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EXECUTIVE  SUMMARY 


An  improvised  nuclear  device  (IND)  detonation  in  an  urban  area  would  be  one  of  the  most 
catastrophic  incidents  that  could  occur  in  the  United  States,  resulting  in  immense  loss  of  life  and  damage 
to  property.  The  extreme  scale  of  the  incident,  the  number  of  operations  that  must  be  executed 
concurrently,  and  the  presence  of  radiation  combine  to  create  a  challenging  response  with  unprecedented 
complexity.  All  levels  of  government,  private  organizations,  and  the  general  public  will  face  a  difficult 
response  and  recovery  because  the  enormity  of  the  incident  will  quickly  exhaust  all  available  resources 
[1]  [2]. 

Emergency  managers  of  all  types  will  face  difficult  and  high-impact  choices  throughout  the 
response  effort.  Ensuring  that  emergency  managers  are  prepared  to  make  those  decisions  is  crucial  to 
properly  managing  the  consequences  of  the  incident.  Such  preparation  may  involve  more  than  technical 
training  and  resources,  and  it  may  extend  to  being  cognitively  and  emotionally  prepared  for  the  situations 
emergency  managers  may  face.  Selecting  and  training  those  emergency  managers  is  of  high  importance  to 
the  preparation  effort  and  scrutiny  should  be  applied  to  gaps  that  may  exist  in  current  techniques  for  that 
training  and  evaluation.  To  foster  the  next  generation  of  emergency  management  professionals,  new 
techniques  may  be  required  to  develop  effective  behavioral  competencies,  expedite  the  learning  process, 
and  provide  measurable  performance  metrics  to  select  candidates  who  exhibit  proficiencies  pertinent  to  a 
specific  position. 

MIT  Lincoln  Laboratory  is  engaged  in  a  project  [3]  sponsored  by  the  Department  of  Homeland 
Security  Science  and  Technology  Directorate  (DHS  S&T),  and  the  Federal  Emergency  Management 
Agency  (FEMA)  that  examines  alternative  mechanisms  for  training  and  evaluation  of  emergency 
managers  to  augment  and  complement  existing  techniques.  The  effort  seeks  to  answer  the  question  of 
how  government  agencies  can  ensure  that  key  emergency  response  personnel  have  the  required  skills  and 
knowledge  to  make  critical  decisions  during  an  incident  of  unprecedented  size,  scope,  and  complexity 
such  as  an  IND  detonation. 

This  document  reports  on  findings  and  analysis  during  the  first  phase  of  the  project,  which  was  to 
gain  an  understanding  of  the  specific  response  decisions  and  decision  making  skills  required  during  an 
IND  detonation  response.  The  project  team  interviewed  and  surveyed  emergency  response  professionals, 
analyzed  the  decisions  they  emphasized  as  being  both  important  and  difficult,  and  analyzed  the  associated 
skills  they  identified  as  being  critical  to  making  these  decisions.  The  remainder  of  the  report  provides 
detail  on  the  method  and  findings  of  these  interviews.  This  effort  does  not  intend  to  develop  an  exhaustive 
list  of  all  relevant  decisions  and  skills.  Rather,  it  aims  to  prioritize  key  skills  and  focus  attention  on 
potential  gaps,  in  order  to  lay  the  groundwork  for  enhancements  to  training  and  evaluation  techniques. 


The  key  findings  from  the  phase  one  analysis  are  as  follows: 

•  There  is  strong  consistency  in  both  the  key  decisions  and  underlying  skills  emphasized  by 
emergency  response  professionals  required  for  an  IND  incident. 

•  The  skills  emphasized  as  gaps  by  those  interviewed  were  largely  “soft  skills,”  defined  as 
cognitive  and  interpersonal  proficiencies  that  are  not  directly  addressed  by  existing  training  and 
evaluation  mechanisms. 

•  We  observed  trends  in  the  “soft  skills”  most  valued  by  different  types  of  emergency  managers 
that  we  interviewed. 

•  Based  on  an  analysis  of  emergency  management  decision  and  skill  requirements,  virtual 
training  methods  (“serious  games”),  including  traditional  scenario  driven  training  exercises  and 
lighter-weight  games  in  the  style  of  modem  entertainment  platforms,  could  be  developed  and 
leveraged  to  improve  local,  state,  and  federal  response  capabilities  for  large  scale,  infrequent, 
and  complex  incidents  like  an  improvised  nuclear  device  detonation. 
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1.  PROJECT  OVERVIEW 


The  overall  project  [3]  addresses  the  question  of  how  government  agencies  can  ensure  that 
emergency  managers  have  the  appropriate  knowledge,  skills,  and  abilities  (KSAs)  to  operate  effectively 
during  an  incident  of  unprecedented  size,  scope,  and  complexity,  such  as  an  IND  detonation.  To  help 
answer  this  question,  we  are  exploring  the  use  of  gaming  as  a  skill  enhancement  and  evaluation  technique 
to  target  “soft  skills,”  which  include  personality  traits,  emotional  intelligence,  critical  thinking,  and  inter¬ 
personal  skills,  as  well  as  improving  the  retention  of  training  “hard  skills,”  such  as  technical  knowledge, 
familiarity  with  best  practice,  and  knowledge  about  an  organization’s  structure  and  operation.  The  project 
evaluates  the  viability  of  expanding  the  use  of  serious  games  to  augment  classroom  training,  tabletop  and 
full  scale  exercise,  and  actual  disaster  experience  to  better  cover  the  critical  skills  needed  for  effective 
emergency  management.  This  report  documents  the  first  phase  of  the  project:  analyzing  the  key  decisions 
and  skills  relevant  to  the  response  effort  for  an  IND  detonation.  Subsequent  phases  of  the  project  will 
build  on  this  catalog  of  decisions  and  skills  to  evaluate,  design,  and  integrate  new  training  and  evaluation 
mechanisms  that  could  better  prepare  emergency  managers  to  respond  to  catastrophic  incidents. 

Serious  games  are  used  throughout  DHS  and  other  government  agencies  to  support  training, 
evaluation,  analysis,  and  technology  exploration.  Those  techniques  have  found  successful  niches,  but  their 
wider  applicability  faces  several  practical  barriers.  Those  barriers  can  potentially  be  overcome  by  drawing 
on  techniques,  technologies,  and  expertise  from  the  entertainment  game  industry.  The  recent  surge  of 
activity  in  attempting  to  do  so  has  had  successes  [4]  [5],  but  is  hampered  by  the  lack  of  a  systematic 
approach  for  reliably  building  games  to  match  a  given  application. 

1.1  LEARNING  CRITICAL  SKILLS 

It  is  difficult  and  time  consuming  to  gain  expertise  within  a  specific  job  or  profession  when  the 
required  skills  are  challenging  to  learn,  and  opportunities  to  gain  actual  experience  are  infrequent.  This 
challenge  is  especially  prevalent  in  the  field  of  radiological  and  nuclear  (rad/nuc)  emergency 
management,  where  real-world  instances  are  extremely  rare  and  many  details  are  theoretical.  Currently, 
emergency  management  personnel  hone  their  professional  competencies  through  a  combination  of 
classroom  and  on-the-job  learning.  Classroom-based  learning  provides  professionals  with  the 
fundamentals  of  their  position  requirements,  focusing  on  gaining  technical  knowledge,  response 
terminology,  and  incident  organization  fundamentals,  including  the  National  Incident  Management 
System  (NIMS)  and  Incident  Command  System  (ICS).  On-the-job  learning  and  experience-based  job 
requirements  help  ensure  that  personnel  have  learned  relevant  skills  by  successfully  tackling  real 
problems  and  challenges. 

The  combination  of  classroom  and  on-the-job  learning  can  be  effective  and  can  establish  expertise 
and  skill;  however  there  are  shortcomings  with  each  method  individually,  and  even  with  both  methods 
combined.  Classroom-style  methods  struggle  to  teach  and  help  students  retain  abstract  behavioral 
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competencies.  On-the-job  learning  can  be  highly  effective,  but  for  complex  professions  like  emergency 
management  it  can  be  a  slow  method  of  gaining  expertise,  especially  when  incidents  are  rare  (as  is  the 
case  for  large  scale  disasters).  Some  incident  commanders  and  senior-level  emergency  managers  spend 
decades  becoming  proficient.  On-the-job  learning  is  challenging  to  quantify  in  terms  of  improved  skill 
and  performance,  and  it  is  hard  to  determine  if  experience  and  skills  gained  in  one  type  of  situation 
transfer  to  similar  situations  (for  example,  between  disasters  of  different  types,  of  different  scales,  or  in 
different  locations).  To  be  better  prepared  for  a  catastrophic  response,  new  and  innovative  methods  are 
needed  to  develop  emergency  managers’  decision  making  skills. 

The  nature  of  the  skills  to  be  learned  complicates  the  task  of  cultivating  skilled  experts.  Within 
emergency  management  as  a  whole,  the  subjects  of  some  of  our  interviews  postulated  that  the 
distinguishing  competencies  of  successful  personnel  are  more  behavioral  than  knowledge-based;  the 
analysis  described  in  this  report  supports  this  speculation.  Usually,  the  most  adept  emergency  managers 
and  incident  commanders  possess  a  refined  set  of  behavioral  competencies,  such  as  the  ability  to 
improvise,  make  correct  inferences,  and  to  selectively  extract  useful  information  to  enhance  their 
situational  awareness.  Such  skills  can  be  built  up  through  lengthy  experience,  or  are  perhaps  innate,  but 
they  are  especially  hard  to  develop  and  demonstrate  in  the  classroom,  or  via  infrequent  field  experience. 
Waiting  for  candidate  emergency  managers  to  slowly  acquire  and  demonstrate  the  skills  during  real 
incidents  curbs  the  available  talent  pool,  and  reduces  confidence  that  key  decision  makers  have  the  skills 
to  handle  a  large  incident. 

Motivating  this  project  is  a  set  of  research  questions  arising  from  the  discussion  above  that  skilled 
emergency  managers  are  successful  because  they  possess  certain  behavioral  competencies.  The  most 
interesting  research  questions  are  listed  below: 

1.  Do  successful  emergency  managers,  especially  those  with  a  lengthy  track  record  of  successful 
field  experience,  actually  value  and  exhibit  a  certain  set  of  behavioral  competencies,  and  are 
these  skills  common  across  types  of  disasters? 

2.  Which  skills  are  the  most  important  to  each  level  or  type  of  emergency  manager,  and  what  are 
the  important  hard  decisions  that  require  those  skills? 

3.  Are  there  mechanisms  to  reliably  determine  if  someone  already  possesses  those  critical  skills, 
or  what  potential  they  exhibit  to  acquire  such  skills? 

4.  Can  the  learning  of  those  skills  be  accelerated  or  better  retained  through  the  use  of  non- 
traditional  training  mechanisms,  such  as  serious  games  or  game-like  exercises? 

The  interview  and  survey  analysis  described  in  this  report  target  research  questions  one  and  two 
above,  laying  the  groundwork  for  subsequent  phases  of  the  project  to  address  new  evaluation  and  training 
techniques  described  in  the  following  subsection. 
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1.2  SERIOUS  GAMES  AT  THE  DEPARTMENT  OF  HOMELAND  SECURITY  (DHS) 


While  there  is  still  organization  stigma  attached  to  the  word  “gaming”  because  it  is  often  associated 
with  entertainment,  serious  games  are  already  widely  used  throughout  DHS  and  the  Department  of 
Defense  (DoD)  to  improve  employees’,  responders’,  and  warfighters’  skills  and  capabilities  to  execute 
their  mission.  Serious  games  are  currently  used  in  the  form  of  immersive  simulations,  live  exercises, 
scenario  based  training,  red  teaming,  and  war  gaming.  For  example: 

•  DHS  Transportation  Security  Administration  uses  a  rehearsal  game  to  train  airport 
checkpoint  screeners  to  quickly  spot  dangerous  items,  to  identify  their  best  employees,  and  to 
provide  truth  data  for  training  automated  detection  algorithms.  [6] 

•  Airline  pilots  and  air  traffic  controllers  gain  valuable  flight  experience  from  detailed 
simulators,  reducing  the  cost  of  gaining  experience  by  augmenting  true  flying  time,  and 
demonstrating  their  skills  in  a  low  risk  environment. 

•  Emergency  responders  come  together  to  participate  in  scenario-driven  exercises  to  practice 
coordinating  and  communicating,  and  to  build  organizational  ties  and  social  networks  of  trust. 

•  Naval  officers  evaluate  the  relative  effectiveness  of  different  command  structures  and  the 
potential  uses  of  emerging  technologies  through  war  games,  using  a  mix  of  virtual  and  tabletop 
materials.  [7] 

•  Cyber  security  analysts  often  use  a  competitive  exercise  environment  with  a  red  team 
attempting  to  find  exploits  in  a  blue  team’s  defenses,  essentially  crowdsourcing  human 
ingenuity  for  finding  gaps. 

All  of  these  approaches  are  types  of  serious  games,  which  can  be  defined  as  any  interactive  system 
that  replicates  a  key  dynamic  or  decision  space  from  the  real  world  in  a  controlled,  artificial  environment 
that  focuses  participants  on  a  particular  aspect  of  the  domain.  Games  provide  success  criteria  to  motivate 
participants  to  learn,  exhibit,  or  explore  skills  and  situations  selected  by  the  designer,  and  of  interest  to 
analysts.  These  features  of  games  can  be  exploited  to  reduce  the  cost  of  training,  supplement  real  world 
experience,  and  target  skills  and  situations  that  cannot  be  adequately  captured  in  the  classroom. 

While  existing  gaming  techniques  are  effective  within  their  specific  applications,  there  is  a  potential 
for  a  much  broader  positive  impact.  That  broader  impact  is  currently  limited  by  three  key  challenges 
facing  potential  users: 

CHALLENGE  1:  Historical  niches  and  focus  on  rehearsal.  Most  current  serious  games  focus  on 
rehearsal;  participants  walk  through  the  motions  of  known  procedures  and  best  practices.  Rehearsal 
is  a  very  important  part  of  preparedness,  but,  as  our  analysis  indicates,  there  are  other  important 
skills  to  target  at  both  the  individual  and  organization  level.  For  example,  dealing  with  unexpected 
situations,  experimenting  with  alternative  organizational  structures,  and  building  social  networks  of 
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trust  are  sometimes  addressed  by  current  games,  and  anecdotally  games  that  target  those  topics  are 
very  helpful,  but  generally  take  a  back  seat.  Naval  war  gaming  has  a  tradition  of  emphasizing  such 
dynamics  [7],  but  those  practices  are  not  frequently  employed  by  DHS.  This  is  understandable,  as  it 
would  be  difficult  for  an  emergency  response  exercise  designer  to  adapt  a  naval  war  game  to  their 
situation  using  current  tools  and  techniques.  A  more  systematic  and  data-driven  understanding  of 
serious  games  could  offer  a  means  for  identifying  when  a  game  is  an  appropriate  tool  to  use,  which 
type  of  game  would  best  suit  the  application,  and  provide  guidance  for  adapting  previous  games  to 
new  applications. 

CHALLENGE  2:  Lack  of  reusable  templates  and  infrastructure.  The  design,  construction,  and 
analysis  of  serious  games  are  generally  ad  hoc  and  do  not  draw  on  a  common  set  of  tools  or 
techniques.  New  games  are  generally  created  bottom-up,  both  in  terms  of  their  design  and  their 
implementation,  even  when  they  are  quite  similar  to  previous  games.  A  catalog  of  game  templates 
coupled  with  matching  reusable  infrastructure  would  reduce  the  cost  of  creating  new  games  or 
adapting  old  games  to  new  situations. 

CHALLENGE  3:  High  operation  and  attendance  costs.  Serious  games  currently  in  use  are 
heavy  weight;  they  generally  require  specialized  equipment,  a  considerable  time  commitment  by 
participants  and  designers,  and  lengthy  manual  analysis  of  the  outcomes.  Simulations  generally 
require  participants  to  visit  a  particular  location  to  use  specialized  equipment,  limiting  the  volume 
of  data  that  can  be  collected  (to  determine  aggregate  patterns  and  trends).  Live  exercises  often 
require  participants  to  travel  to  a  common  location  and  remain  there  for  several  days  in  a  row, 
increasing  the  cost  of  such  training,  and  reducing  the  chance  that  an  agency  will  simultaneously 
send  all  its  best  personnel  for  training.  There  is  an  opportunity  for  additional  styles  and  alternate 
implementations  of  serious  games  to  allow  asynchronous  or  remote  participation,  to  require  less 
specialized  equipment  to  operate,  to  better  target  specific  skills  or  situations  of  interest,  and  to 
overall  place  less  burden  on  the  participants  and  designers. 

The  entertainment  gaming  industry  has  faced  and  overcome  a  similar  set  of  obstacles,  and  many  of 
their  solutions  could  apply  to  the  realm  of  serious  games.  There  are  several  elements  of  the  gaming 
industry  innovation  that  could  be  better  leveraged  to  support  serious  gaming  at  DHS: 

•  A  language  to  describe  and  compare  game  archetypes,  player  experience,  and  design  tradeoffs. 

•  Concepts  and  methods  for  selecting  an  appropriate  archetype  for  a  given  audience  or  to  create 
desired  dynamic  between  players. 

•  Tools  and  infrastructure  for  creating  compact  deployable  games  that  can  be  run  on  ordinary 
desktop  machines  and  create  interaction  between  players  at  remote  locations. 

•  Techniques  for  increasing  player  engagement  and  motivating  players  to  pursue  in-game 
objectives. 
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Experience  logging  and  analyzing  player  data  to  quantitatively  measure  engagement,  calibrate 
difficulty,  and  tune  in-game  incentives. 


•  Existing  companies  with  expertise  to  design,  build,  and  deploy  games.  Game  design  companies 
exist  in  many  sizes,  from  large  “triple  A”  companies  with  thousands  of  employees  all  the  way 
down  to  independent  developers.  They  range  from  companies  dedicated  to  entertainment,  to 
those  targeting  businesses  for  ongoing  education. 

In  recent  years,  there  has  been  a  flurry  of  activity  surrounding  building  lighter-weight  serious 
games  in  the  style  of  entertainment  games  [8].  There  are  multiple  companies  offering  game-based  training 
and  development  support  for  serious  game  projects.  The  general  consensus  of  the  research  and 
commercial  communities  is  that  such  games  have  been  demonstrated  to  be  capable  of  teaching  both  hard 
and  soft  skills,  and  transferring  those  skills  to  other  domains  [4]  [5]  [9].  However,  there  is  still  limited 
understanding  of  how  to  reliably  create  a  game  to  target  a  particular  skill  or  application  [10].  It  is  also 
difficult  to  determine  the  efficacy  of  a  given  serious  game  without  extensive  and  expensive  human 
studies.  Without  a  systematic  method  for  selecting  an  appropriate  style  of  game  for  a  given  application, 
designers  are  operating  in  the  dark  and  can  only  produce  effective  games  via  a  fairly  expensive  trial  and 
error  process. 
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2.  PROJECT  MISSION 


This  project  seeks  to  analyze  potential  carry-overs  from  the  commercial  gaming  industry  and 
determine  their  applicability  to  addressing  existing  needs  and  gaps  within  DHS.  To  accomplish  this  task, 
the  project  aims  to: 

•  Identify  an  expanded  set  of  challenges  within  DHS  that  could  be  addressed  with  games. 

•  Begin  to  build  a  systematic  classification  of  game  styles  and  quantifiably  match  those  styles  to 
suitable  applications. 

•  Understand  the  types  of  collaborations  between  DHS  and  private  industry  that  could  create 
games  to  directly  support  DHS  needs. 

The  project’s  initial  focus  is  on  an  IND  detonation  as  a  concrete  example  of  a  DHS  mission  area 
where  serious  gaming  is  likely  to  be  productively  employed.  An  IND  detonation  would  stress  DHS  and 
emergency  response  systems  and  resources,  so  an  analysis  of  the  relevant  decisions  and  skills  of  such  an 
incident  is  likely  to  identify  areas  of  need  not  currently  being  sufficiently  met  by  existing  training  and 
preparation  methods.  Furthermore,  IND  incidents  are  low  probability,  high  consequence  incidents, 
making  preparation  important  but  difficult  to  justify  when  it  is  costly.  Thus,  INDs  represent  an 
opportunity  for  lighter-weight  games  to  play  a  valuable  role  in  preparation.  Lastly,  our  analysis  indicates 
that  an  IND  response  effort  requires  largely  the  same  skills  as  other  emergency  response  but  with  unique 
challenges.  The  commonality  to  other  incidents  means  that  the  outcomes  of  this  project  are  likely  to  apply 
more  widely,  while  the  unique  elements  will  drive  the  work  to  generate  more  general  game  templates  (not 
just  one-off  games)  that  are  flexible,  and  that  can  readily  be  adapted  to  other  scenarios. 
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3.  PHASE  ONE:  ANALYSIS  OF  IND  INCIDENT  RESPONSE  DECISION 
MAKING  SKILL  CHARACTERISTICS 


3.1  OVERVIEW 

The  first  step  in  designing  training  and  evaluation  mechanisms  for  any  skill  set  is  to  identify  the 
important  decisions  and  corresponding  skills.  Since  no  IND  incidents  have  occurred  in  the  US,  there  is  no 
direct  way  to  identify  those  decisions  and  skills.  As  a  proxy  for  direct  experience  with  an  IND,  we  turned 
to  hazard  and  planning  documentation  and  subject  matter  experts  (SMEs).  Existing  hazard  and  planning 
documentation  covers  the  nature  of  radiation  and  response  plans;  however  our  primary  source  of 
information  for  the  key  decisions  and  skills  required  was  SMEs  with  experience  creating  IND  incident 
response  plans  and  participating  in  non-IND  large  scale  incidents. 

This  analysis  is  not  designed  to  be  a  comprehensive  guide  to  an  IND  response.  The  focus  of  this 
phase  of  the  project  is  to  identify  trends  in  the  decisions  and  skills  SMEs  emphasized  and  what  concerns 
they  expressed  about  current  preparedness.  These  results  and  our  interpretations  are  presented  in  the 
following  sections,  laying  the  groundwork  for  subsequent  phases  of  the  project. 

3.2  METHODOLOGY 

Figure  1  outlines  the  process  we  took  to  complete  phase  one  of  this  work.  We  started  by  reviewing 
relevant  IND  incident  response  planning  documents,  including: 

•  Planning  Guidance  for  Response  to  a  Nuclear  Detonation  [1] 

•  FEMA’s  IND  Response  Planning  Tool  [11] 

•  Key  Planning  Factors  for  Response  to  a  Nuclear  Detonation  in  Houston,  Texas  [12] 

•  National  Response  Framework  Radiological/Nuclear  Incident  Annex  [13] 

•  DHS  Strategy  for  Improving  the  National  Response  and  Recovery  from  an  IND  Attack  [14] 
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Reviewed  documents  to  understand  IND  response 
preparedness 


k  i 


Interviewed  SMEs  to  identify  decisions  and  skill 
requirements 


Documented  critical  decisions  and  skills 


Validated  critical  decisions  and  skills  through  additional 
interviews 


Figure  1.  Phase  one  methodology’. 


Next  we  interviewed  26  SMEs  in  the  fields  of  large-scale  incidents  and  IND  incident  response 
planning.  Our  interviews  were  mainly  with  FEMA  Incident  Management  Assistance  Team  (IMAT) 
emergency  managers,  state-level  emergency  managers,  federal  level  IND  planning  specialists,  state  level 
IND  planning  specialists,  and  radiological  health  experts.  Table  1  includes  a  summary  of  the  types  of 
SMEs  that  we  interviewed.  The  SMEs  have  extensive  experience  in  large-scale  emergency-response 
decision  making  at  the  local,  state,  and  national  levels.  The  list  includes  high-level  coordinating  officers 
from  many  notable  major  disasters  such  as:  9/11  (2001),  Hurricane  Katrina  (2005),  Haiti  Earthquake 
(2010),  and  Hurricane  Sandy  (2012).  While  there  is  a  wide  range  of  people  that  would  be  involved  in  an 
IND  detonation  incident  response  beyond  those  that  were  targeted  for  this  study,  we  believe  that  the 
group  of  SMEs  chosen  for  this  study  is  representative  of  those  making  the  most  critical  decisions. 
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Table  1 

Summary  of  26  SMEs  Interviewed 


Organization 

Role(s) 

Association  of  State  and  Territorial  Health  Officials 

Public  Health 

National  Nuclear  Security  Administration 

Health  Physicist 

FEMA 

IMAT  Officers,  Planning,  Command  Staff, 

Radiation  Communication 

Lawrence  Livermore  National  Laboratories 

Health  Physicist 

New  Jersey  State  Police  Office  of  Emergency 
Management 

Radiological  Officer,  Planning,  Command  Staff 

New  Jersey  Office  of  Homeland  Security  and 
Preparedness 

Planning 

FDNY  Center  for  Terrorism  and  Disaster 

Preparedness 

Command  Staff 

NYC  Department  of  Health  and  Mental  Hygiene 

Planning 

Department  of  Health  and  Human  Services 

Public  Health 

Centers  for  Disease  Control  and  Prevention 

Health  Physicist 

We  concluded  the  interview  process  when  we  no  longer  heard  new  topics  from  the  interviewees, 
giving  us  reasonable  confidence  that  the  quantity  of  interviews  conducted  was  sufficient  to  represent  the 
dominant  views  of  this  type  of  emergency  manager.  As  described  later  in  this  report,  there  were  strong 
consistencies  in  the  topics  that  the  SMEs  choose  to  emphasize  when  asked  open-ended  questions. 

The  interviewees  responded  to  questions  by  phone  and  the  interview  sessions  lasted  one  hour.  The 
majority  of  the  interviews  had  a  single  SME  present;  two  interviews  had  two  SMEs  each.  For  those  two 
interviews,  the  opinions  of  the  two  SMEs  were  combined  into  a  single  entry  in  our  results  in  Section  3.  A 
project  summary  similar  to  Section  1  of  this  report  was  provided  to  each  SME  ahead  of  time.  The 
majority  of  the  interview  time  included  a  discussion  about  the  difficult  decisions,  and  required  skills  in 
the  context  of  1ND  and  large  scale  incidents.  The  discussion  had  a  few  opened-ended  questions,  but  the 
bulk  of  the  time  was  open  for  the  interviewee  to  discuss  any  topics  relating  to  rad/nuc  response  of  their 
choosing.  Examples  of  the  open-ended  questions  are 

•  What  are  difficult  decisions  or  activities  in  an  IND  or  large-scale  incident? 

•  What  are  difficult  positions  or  roles  in  an  IND  or  large-scale  incident? 

•  What  skills  are  most  important  to  those  decisions? 
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•  How  comfortable  are  you  that  the  people  making  these  decisions  have  the  skills  needed? 

•  Which  skills  are  hardest  to  teach,  select  for,  or  validate? 


This  interview  method  encouraged  discussions  that  allowed  each  SME  to  emphasize  the  most 
challenging  and  important  aspects  of  emergency  response  as  they  relate  to  his/her  experiences  and 
concerns.  The  topics  discussed  were  spontaneously  emphasized  by  the  SMEs,  and  not  intended  to  be  a 
comprehensive  list.  The  interviews  were  meant  to  verify,  prioritize,  and  augment  the  topics  discussed  in 
more  comprehensive  reports  (see  the  references  section)  not  to  supersede  planning  documentation. 

Our  final  step  was  to  validate  the  findings  from  the  interviews.  This  was  done  by  surveying  the 
same  group  of  SMEs,  plus  an  expanded  group  of  similar  SMEs  to  rank  the  importance  and  level  of 
concern  of  information  collected  during  the  interview  process.  Answering  questions  on  a  provided  list  of 
topics  reduces  the  chance  that  an  important  topic  was  accidentally  omitted  by  some  SMEs  during  their 
interview.  Note  that  a  ranking  approach  would  not  have  been  appropriate  before  the  interviews,  as 
ranking  a  provided  list  would  not  allow  the  SMEs  to  express  their  own  ideas,  and  could  bias  them  toward 
theories  we  already  had.  However,  once  the  SMEs  provide  a  list  in  an  open-ended  fashion,  a  more 
structured  follow-on  analysis  can  be  useful  and  complementary.  The  survey  specifically  asked  the  SMEs 
to  rank  the  importance  of  items  in  two  lists,  corresponding  to  the  impactful  decisions  and  required  skills 
of  an  IND  incident  response.  When  the  SMEs  ranked  the  items,  they  received  guidance  to  incorporate 
their  opinions  about  two  topics:  the  effectiveness  of  existing  decision  or  skill  training,  and  the  importance 
of  that  decision  or  skill. 

3.3  LIMITATIONS  OF  THIS  ANALYSIS 

Below,  we  outline  the  key  limitations  of  this  analysis  and  discuss  how  they  affect  the  application  of 
our  findings. 

Interpreting  Emphasis  as  Importance 

This  analysis  focused  on  identifying  decisions  and  skills  that  were  emphasized  by  professionals  in 
response  to  open-ended  questions.  We  used  this  approach  to  avoid  biasing  the  responses,  but  at  the  same 
time  it  introduces  a  risk  that  the  SMEs  overlooked  a  topic  during  the  interview  that  they  believe  is 
important.  Therefore,  our  findings  do  not  indicate  the  full  curriculum  for  training  or  criteria  for  evaluating 
emergency  managers,  but  rather  they  identify  and  prioritize  areas  for  improvement  or  augmentation. 
Additionally,  the  frequency  of  mention  during  a  one  hour  interview  might  not  correlate  to  perceived 
importance.  We  mitigated  this  concern  both  by  conducting  a  large  number  of  interviews  (24  interviews 
covering  26  SMEs)  and  by  supplementing  those  interviews  with  a  structured  survey.  The  survey  included 
all  items  mentioned  by  at  least  two  SMEs  and  required  the  participant  to  rank  them  by  importance.  The 
structure  of  the  survey  prevented  a  topic  from  being  ignored  or  forgotten.  The  survey  results  closely 
match  the  interview  results,  with  a  few  exceptions  noted  below  in  Sections  4.1.2  and  4.2.2. 
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Questionnaire  Response  Rate 


The  questionnaire  received  a  less  than  50  percent  response  rate  from  the  SMEs.  Only  10  of  26 
interview  participants  responded,  plus  an  1 1th  response  from  an  SME  not  interviewed.  As  a  result,  the 
survey  qualitatively  helps  to  validate  the  interview  approach,  but  it  does  not  provide  much  data  for 
additional  analysis.  The  bulk  of  our  analysis  and  identified  trends  come  from  the  larger  set  of  interviews. 

Types/Levels  of  Emergency  Managers 

For  this  study,  we  targeted  FEMA  National  IMAT  staff  and  FEMA  regional  staff,  federal-level  IND 
planning  specialists,  state-level  emergency  managers,  IND  planning  specialists,  and  radiological  health 
SMEs.  We  believe  that  the  set  of  interviewees  is  representative  of  those  who  would  have  significant  roles 
in  an  IND  incident  response.  However,  that  range  of  types  also  meant  that  we  had  only  a  handful  of 
participants  from  any  one  category,  and  thus  we  only  report  trends  with  a  high  degree  of  consistency  in 
the  results.  It  is  difficult  to  identify  more  subtle  patterns  with  any  confidence  in  a  small  data  set. 

Although  a  wider  range  of  emergency  managers  could  be  making  influential  decisions,  we  believe 
we  have  at  least  sampled  the  key  decision  makers  within  government  roles.  We  did  not  consider 
nongovernmental  organizations  (NGOs)  or  every  government  agency  that  may  have  a  role  in  an  IND 
incident.  Also,  several  SMEs  pointed  out  that  “the  public  is  the  real  first  responder.”  This  is  especially  the 
case  during  large  scale  incidents  where  survivors  and  members  of  the  affected  public  can  provide  much  of 
the  relief.  We  did  not  address  this  issue  or  how  grass-roots  response  efforts  could  be  made  more  effective, 
either  in  advance  or  during  the  incident. 

Choosing  Skill  Categories 

The  categories  of  skills  we  reported  describe  informal  and  abstract  skills;  as  a  result  categorizing 
the  skills  is  somewhat  arbitrary.  We  grouped  together  skills  if  the  SMEs  gave  similar  examples  and 
justifications,  but  there  are  several  cases  where  skills  could  be  combined  or  split  differently.  Doing  so 
could  alter  the  ranking  of  the  skills,  although  we  believe  it  would  be  only  a  minor  change.  Nonetheless,  it 
may  be  helpful  in  the  future  to  more  thoroughly  compare  the  skill  categories  we  generated  with  those  that 
have  been  used  by  similar  analyses  in  other  fields  [15]  [16]. 

Applying  these  Results  to  Training  and  Evaluation 

The  application  of  this  analysis  to  training  and  evaluation  programs  assumes  that  the  skills  valued 
and  reported  by  current  professionals  are  indeed  the  important  skills  required  of  professionals  in  future 
events.  There  are  four  implicit  assumptions  in  that  approach  that  should  be  made  explicit  and  justified 
before  applying  them  to  the  actual  development  of  training  and  evaluation  materials. 

•  Assumption  1:  Current  emergency  managers  have  the  requisite  skills.  We  believe  this  is  a 
safe  assumption  since  the  majority  of  the  SMEs  we  interviewed  have  extensive  experience  and 
expertise,  which  was  developed  through  years  of  traditional  training,  participation  in  exercises 
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and  drills,  and  success  in  real-life  emergency  response  experience.  The  goal  of  expanded 
training  and  evaluation  techniques  would  be  to  accelerate  the  acquisition  of  skills  that  are 
currently  acquired  over  many  years,  but  our  approach  assumes  that  the  slower  path  does  in  fact 
build  and  demonstrate  those  skills. 

•  Assumption  2:  Current  emergency  managers  with  the  requisite  skills  know  what  those 
skills  are.  The  use  of  interviews  of  SMEs  assumes  that  the  SMEs  know  what  makes  them 
effective.  It  is  possible  for  SMEs  to  possess  skills  without  being  aware  of  them  or  without  being 
able  to  articulate  them.  Our  best  mitigation  of  this  risk  is  to  include  SMEs  with  lengthy 
experience  and  those  who  have  been  involved  in  selecting  and  hiring  other  emergency 
managers.  Cross-checking  the  skills  reported  with  work  on  soft  skills  in  other  domains  [15] 
[16]  will  help  validate  the  results,  but  there  will  always  remain  a  chance  that  the  SMEs  simply 
are  not  aware  of  what  makes  them  effective. 

•  Assumption  3:  The  set  of  important  skills  for  emergency  response  professionals  as  a  whole 
will  remain  the  same  in  the  future.  We  have  no  reason  to  believe  that  the  fundamental  nature 
of  emergency  response  will  change  [1].  The  same  types  of  social  and  political  structures  and  the 
same  types  of  disasters  will  persist  for  the  foreseeable  future,  and  thus  we  believe  that  the  same 
hard  decisions  and  skills  will  be  present. 

•  Assumption  4:  The  skills  relevant  to  non-IND  incidents  carry  over  to  IND  incidents. 

Current  emergency  managers  have  never  experienced  an  IND,  and  can  only  discuss  other  large- 
scale  incidents  and  their  analysis  of  the  likely  nature  of  an  IND.  The  FEMA  IMAT  SMEs  we 
interviewed  were  in  agreement  that  the  key  skills  relevant  to  an  IND  incident  would  be  no 
different  than  those  needed  for  another  large-scale  disaster.  That  belief  is  also  reflected  in 
documents  we  reviewed  [1].  The  SMEs  were  not  explicitly  asked  how  IND  and  non-IND 
incidents  would  compare,  but  many  of  them  provided  explanations.  The  SMEs  seemed  in 
agreement  that  the  scale  of  an  IND  incident  and  the  presence  of  radiation  would  alter  the  best 
response  to  certain  decisions,  but  not  what  those  decisions  were  or  what  skills  would  help 
address  them. 
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4.  RESULTS 


4.1  HIGH  IMPACT  DECISIONS  DURING  AN  IND  INCIDENT  RESPONSE 

Table  2  lists  all  nineteen  decisions  emphasized  by  three  or  more  SMEs  as  being  critical  during  the 
response  to  an  IND  detonation  or  other  large-scale  disasters;  decisions  mentioned  by  only  one  or  two 
SMEs  were  omitted  as  nonrepresentative.  The  rest  of  this  section  provides  a  more  detailed  definition  and 
explanation  for  each  decision  based  on  the  information  and  examples  directly  provided  by  the  SMEs.  The 
level  of  detail  devoted  to  each  item  in  this  section  reflects  the  detail  provided  by  the  SMEs,  the  number  of 
SMEs  who  discussed  the  topic,  and  the  importance  placed  on  the  specific  decision  by  the  SMEs  as  a 
whole  (Figure  2). 

Table  2 

List  of  Decisions  Emphasized  by  Interviewed  SMEs 

Decisions  Emphasized  Page  Number 

How  to  allocate  finite  resources?  16 

How  to  phrase  and  disseminate  information  to  the  public  and  local  first  responders  ?  17 

Whether  to  expose  first  responders  to  radiation  and  to  what  extent?  17 

Where,  when,  and  how  to  evacuate?  18 

How  to  set  patient  triage  standards?  19 

Where  are  the  dangerous  fallout  and  hot  zones?  19 

Where  to  issue  shelter-in-place  recommendation?  20 

How  to  develop  an  information  collection  plan?  20 

Where  and  how  to  stage  incoming  resources?  20 

Where  to  assemble  survivors?  21 

How  to  recognize  as  a  rad/nuc  incident?  21 

Where  are  the  damage  zones?  22 

Has  the  decision-making  staff  been  impacted?  22 

How  to  get  public  services  and  utilities  back  up?  22 

How  and  where  to  decontaminate  people?  22 

How  to  monitor  the  affected  population?  23 

How  to  maintain  public  confidence  and  trust?  23 

Should  first  responders  seal  off  areas  and  restrict  who  can  leave  and  enter?  23 

Is  this  a  single  detonation  or  is  there  a  secondary  device?  23 
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4.1.1  Decision  Descriptions  and  Examples 
How  to  allocate  finite  resources?  (mentioned  by  16  of  24  inteniewees) 

This  decision  was  the  most  frequently  mentioned  and  was  indicated  by  a  majority  of  SMEs 
interviewed  as  being  a  very  difficult  part  of  a  successful  response.  Resource  allocation  came  up  in  a 
number  of  different  contexts  depending  on  the  background  and  experience  of  the  SME,  whether  it  was 
search  and  rescue  (SAR)  supplies/personnel,  medical  supplies/personnel,  debris  clearing  capabilities,  etc. 
The  common  theme  was  that  all  resources  would  be  very  limited  in  an  IND  incident  response  and  that 
decisions  would  need  to  be  made  quickly  in  order  to  maximize  use  in  the  response  efforts. 

A  unique  part  of  an  IND  incident  response,  as  noted  by  many  SMEs,  is  that  the  best  decision  just 
after  detonation  may  often  be  deciding  to  save  the  limited  resources  for  a  later  time  when  they  can  be 
more  impactful.  The  reason  is  related  to  the  health  effects  of  radioactive  fallout  on  both  victims  and  first 
responders.  Many  SMEs  noted  that  for  almost  all  other  disasters  (for  example,  hurricanes  and 
earthquakes),  first  responders  go  to  the  areas  of  greatest  damage  and  destruction  and  help  those  who  are 
most  injured.  However  in  an  IND  detonation,  people  in  the  areas  of  greatest  damage  will  most  likely  not 
survive  even  with  medical  attention  due  to  effects  from 
radiation;  the  area  of  greatest  impact  for  responders  may  be 
areas  of  the  moderate  damage  zone  that  are  unaffected  or 
lightly  affected  by  radiation  [1],  First  responders  who  go  to 
the  aid  of  people  in  the  severe  damage  zone  may  be  harmfully 
exposed  and  become  unavailable  during  the  remainder  of  the 
response.  Some  responders  may  need  to  shelter-in-place  for 
several  hours  before  providing  aid,  when  their  training  and 
instincts  are  to  rush  to  the  scene.  As  a  result  of  this  stark  difference  in  the  nature  of  the  disaster,  an  IND 
detonation  was  said  to  require  a  “180-degree  response,”  meaning  it  requires  nearly  complete  opposite 
planning  and  actions  compared  to  certain  normal  procedures,  and  making  the  allocation  of  response 
resources  a  nonroutine  challenge.  Of  course,  as  other  SMEs  noted,  some  responders  may  go  too  far  in  the 
other  extreme  and  be  afraid  to  enter  light  radiation  areas  where  they  could  safely  operate  for  many  hours 
at  a  time  and  provide  lifesaving  needs.  Thus,  the  resource  allocation  challenge  is  coupled  to  other  key 
decisions,  such  as  whether  to  endanger  first  responders  (3rd  on  the  list)  and  where  the  dangerous  fallout 
and  damages  zones  are  (6th  and  12th  on  the  list,  respectively). 

Another  major  complication  of  this  decision  is  that  the  resources  will  likely  be  extremely  limited 
when  compared  to  the  magnitude  of  the  incident.  Several  SMEs  mentioned  that  an  IND  detonation  in  a 
city  would  be  “the  biggest  disaster  times  10,”  “100  times  more  difficult  than  other  disasters,”  or  “several 
major  disasters  combined.”  SMEs  thought  that  resources  would  quickly  be  diminished  and  innovative 
solutions  would  be  required  to  keep  up  with  the  response  needs. 


Resources  will  likely  be  extremely 
limited  when  compared  to  the 
magnitude  of  an  IND  incident; 
innovative  solutions  will  be  required 
to  keep  up  with  the  response  needs. 
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How  to  phrase  and  disseminate  information  to  the  public  and  local  first  responders?  (mentioned  by 
14  of  24  interviewees) 

The  IND  incident  response  documentation  and  many  of  the  SMEs  stressed  that  the  shelter-in-place 
communication  message  has  the  potential  to  save  the  most  lives  out  of  all  the  rad/nuc  emergency  response 
activities;  therefore  it  must  be  carefully  addressed  to  ensure  maximum  effectiveness.  A  major  challenging 
decision  for  emergency  responders  is  how  to  communicate  effectively  with  the  public  and  local  first 
responders.  The  messages  must  be  received  and  understood  in  a  consistent  manner  with  the  messages 
provided  by  authorities  at  all  levels  of  government  and  the  media. 


This  decision  is  made  up  of  two  distinct  parts,  first  how  to  physically  get  the  messages  to  those  who 
need  to  be  contacted,  and  second  how  to  word  the  messages  so  that  they  will  be  interpreted  as  desired. 
The  difficulty  of  how  to  get  the  messages  to  those  in  need 
results  from  the  significant  impact  on  communications 
infrastructure  from  the  rad/nuc  detonation;  cellular  towers 
and  public  announcement  systems  may  be  inoperative.  Next, 
the  messages  must  be  crafted  in  such  a  way  to  inspire 
confidence  from  the  public  in  the  plan.  Many  SMEs 
emphasized  that  the  public’s  knowledge,  and  possibly  even 

local  first  responder  knowledge,  of  the  radiation  hazard  and  its  effects  is  minimal;  convincing  both  groups 
that  they  must  follow  instructions  that  may  seem  counterintuitive  will  be  difficult.  The  SMEs  emphasized 
the  importance  of  consistent  guidance  to  avoid  discrediting  all  sources  and  losing  the  ability  to  influence 
behavior.  The  exception  some  SMEs  noted  are  regions  that  have  a  nuclear  power  plant,  where  first 
responders  and  the  public  have  prepared  for  the  appropriate  response. 


Public  messaging  during  an  IND  is 
critical  because  protective  actions 
can  save  lives,  but  will  be  difficult  to 
distribute  because  of  damage  to 
communications  infrastructure. 


These  communication  issues  are  greatly  aggravated  by  two  other  factors,  which  we  introduced  here 
and  which  are  separately  included  in  the  key  decisions  list.  First,  who  are  the  emergency  managers  who 
would  be  gathering  information  and  ultimately  choosing  to  inform  the  public?  In  a  significant  incident 
like  an  IND  detonation  the  emergency  managers  and  decision  makers  may  be,  themselves,  impacted  by 
the  incident.  Second,  it  may  take  some  time  before  the  incident  can  be  verified  as  a  rad/nuc  incident. 
Since  the  initial  shelter-in-place  communication  for  both  the  public  and  local  responders  must  happen 
almost  immediately  after  the  detonation  (on  the  order  of  minutes  not  hours)  to  be  effective  in  preventing 
severe  radiation  exposure,  there  is  little  time  available  to  sort  out  these  issues. 


Whether  to  expose  first  responders  to  radiation  and  to  what  extent?  (mentioned  by  15  of  24 

interviewees) 


The  third  most  frequently  stated  difficult  decision  relates  to  exposure  to  radiation  by  first 
responders.  The  unique  aspects  of  dealing  with  radiation  generate  a  number  of  complicating  factors.  This 
decision  is  difficult  because  responders  are  put  in  danger  by  operating  in  areas  with  radioactive  fallout, 
which  are  generally  the  areas  where  most  of  the  damage  has  occurred.  First  responders  accept  risks  to 
themselves  as  part  of  their  obligations,  but  in  an  IND  incident  there  may  be  a  large  number  of  survivors 
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Responders  are  put  in  danger  by 
operating  in  areas  with  radioactive 
fallout,  which  are  generally  the  areas 
where  most  of  the  damage  has 
occurred  and  help  is  required. 


who  can  be  helped  without  exposing  the  responders  to 
dangerous  levels  of  radiation.  Emergency  managers  will 
have  to  balance  the  ethical  considerations  of  exposing  first 
responders  to  potentially  harmful  radiation  doses,  with 
helping  victims  whose  lives  can  be  saved  with  first 
responder  aid.  The  SMEs  stated  that  it  would  be  difficult  to 
set,  communicate,  and  enforce  decisions  regarding  radiation 


exposure  to  responders.  Additional  questions  related  to  this  decision  posed  by  the  SMEs  are: 


•  How  are  the  exposure  levels  set  to  save  the  most  lives  without  excessively  endangering 
responders?  What  operating  hours  will  be  allowed  in  each  zone? 

•  What  if  some  responders  are  not  comfortable  with  their  command’s  decision,  either  wanting  to 
take  on  more  or  less  risk?  Will  they  be  prevented  from  exposing  themselves  to  undue  risks? 

•  How  will  responders  be  informed  of  procedures  to  maintain  their  own  safety  based  on  their 
operating  location  (e.g.,  number  of  hours  they  can  safely  operate),  and  of  the  dangers  of  not 
following  radiation  safety  instructions? 

•  Should  responders  be  rotated  between  assignments  to  reduce  their  dose  accumulation,  or  will 
that  require  an  unrealistic  level  of  coordination  and  logistics,  and  reduce  overall  effectiveness 
of  the  response? 


The  complexity  of  all  of  these  questions  is  likely  compounded  by  the  lack  of  validated  information 
on  the  boundaries  of  the  fallout  zone  early  in  the  response. 


Where,  when,  and  how  to  evacuate?  (mentioned  by  11  of  24  interviewees) 


This  decision  was  emphasized  as  difficult  due  to  the  uncertainty  of  information  about  the  locations 
of  the  damage,  dangerous  fallout,  and  hot  zones,  as  well  as  the  uncertainty  of  the  infrastructure  available 
for  transporting  people.  An  evacuation  of  an  area  affected  by  fallout  is  most  effective  within  a  window  of 
opportunity — not  so  soon  that  individuals  are  exposed  to  high  doses  during  the  process,  but  not  too  late 
that  they  accumulate  unnecessary  dose  as  they  shelter-in¬ 
place  for  extra  time.  The  dose  accumulated  during  the 
evacuation  will  depend  on  the  level  of  radiation  and  the 
speed  of  the  evacuation — two  factors  that  will  not  be 
known  with  certainty.  The  SMEs  said  it  would  be  difficult 
to  predict  movement  times  of  people  due  to  issues  such  as 
damaged  roadways.  The  following  questions  were 
identified  by  the  SMEs  that  will  be  pertinent  during  an 
evacuation  decision: 


There  is  uncertainty  of  information 
about  the  damage  locations  and 
dangerous  fallout  zones,  combined 
with  uncertainty  of  the  infrastructure 
available  for  transporting  people. 
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•  Has  flash  blindness  caused  excessive  accidents  on  roads  and  caused  them  to  be  impassible?  In 
general,  which  roads  are  clear  or  can  be  cleared  quickly? 

•  Do  people  have  somewhere  to  go  after  they  evacuate,  or  will  they  need  to  be  sheltered  and 
supplied? 

•  Is  there  enough  transportation  capability,  including  personal  vehicles  and  public  transportation, 
to  accommodate  the  evacuation? 

•  How  willing  or  likely  is  the  population  to  adhere  to  evacuation  orders? 

•  Are  decontamination  stations  in  place?  If  not,  should  first  responders  delay  the  evacuation  until 
appropriate  decontamination  locations  are  in  place? 

How  to  set  patient  triage  standards?  (mentioned  by  10  of  24  interviewees) 

This  was  described  by  SMEs  as  a  difficult  ethical  and  risk/reward  management  decision.  In  a 
rad/nuc  incident  many  injured  people  who  may  appear  to  be  savable  are  actually  ill-fated  due  to  their 
excessive  radiation  exposure.  Expending  resources  to  comfort  or  treat  injuries  of  these  individuals  will 
limit  the  resources  available  for  those  individuals  who 

can  be  successfully  treated.  Reorganizing  the  patient  ...  ,  , 

,  ,  .  .  .  ..  Rad/nuc  incidents  are  particularly 

triage  standards  to  include  radiation  exposure  as  a 

. ,  ,  challenging  because  radiation  presence 

consideration  is  very  different  than  medical  treatment  m 
,  .  .  , .  „  ....  „  cannot  be  detected  without  specialized 

other  disasters,  and  is  an  ethically  ditticult  decision  tor 

.  ...  .  .  monitoring  equipment. 

those  providing  treatment  since  there  are  not  precise 

methods  to  determine  the  radiation  exposure  of  patients. 

Furthermore,  medical  responders  may  differ  on  whether  their  goal  is  to  save  lives  in  the  long  run,  or 
reduce  suffering  of  all  affected  individuals. 

Where  are  the  dangerous  fallout  and  hot  zones?  (mentioned  by  9  of  24  interviewees) 

Understanding  the  location  of  radiological  fallout  is  a  significant  concern  for  emergency  managers 
in  planning  and  executing  response  operations.  SMEs  stated  that  rad/nuc  incidents  are  particularly 
challenging  because  radiation  presence  cannot  be  detected  without  specialized  monitoring  equipment. 
Where  radiation  monitoring  equipment  is  available  to  first  responders,  they  will  only  produce  spot 
measurements.  Aggregating  many  of  these  spot  measurements  will  allow  the  mapping  of  the  radiological 
hazards,  but  with  relatively  high  uncertainty  due  to  the  extrapolation  and  interpolation  required  to  create 
maps  from  relatively  sparse  spot  measurements.  Additionally,  the  radiation  levels  will  be  dynamic  and 
change  over  time  based  on  natural  decay  (half-life)  and  meteorological  conditions.  There  are  several 
aspects  of  this  decision  that  were  mentioned: 

•  Do  responders  know  where  the  fallout  zones  are? 

•  Do  responders  understand  the  implications  to  their  safety  and  health  in  the  fallout  zones  from 
the  radiation  levels?  For  example,  will  first  responders  be  willing  to  enter  areas  with  dangerous 
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fallout  without  first  understanding  the  risks,  or  too  hesitant  to  enter  a  hot  zone  even  for  short 
periods? 

•  Do  responders  know  how  to  operate  their  detection  equipment  to  identify  areas  of  high 
radiation  and  measure  or  estimate  their  own  cumulative  doses? 

•  Do  responders  understand  what  levels  of  protection  from  radiation  personal  protective 
equipment  provides  them  in  the  fallout  zone? 

Where  to  issue  shelter-in-place  recommendation?  (mentioned  by  9  of  24  interviewees) 

Similar  to  the  previous  decision  on  where,  who,  and  how  to  evacuate  is  the  issue  of  who  to  instruct 
to  shelter-in-place.  This  decision  was  emphasized  as  difficult  due  to  the  uncertainty  of  information  about 
the  locations  of  the  damage  and  fallout  zones  as  well  as  the  issues  surrounding  the  communication 
problems  mentioned  above.  Further  complicating  issues  mentioned  were: 

•  Where  do  people  go  to  shelter  if  they  do  not  live  in  the  area? 

•  Flow  will  people  react  to  this  guidance,  and  will  they  adhere  to  it? 

•  Flow  do  first  responders  stop  people  from  putting  themselves  at  risk  by  leaving  their  sheltered 
locations  to  reunite  with  loved  ones? 

How  to  develop  an  information  collection  plan?  (mentioned  by  9  of  24  interviewees) 

A  main  theme  of  many  of  the  difficult  decisions  is  that  they  must  be  made  with  limited  information, 
much  of  which  cannot  be  validated.  As  a  result,  prioritizing  and  gaining  more  information  can  be  one  of 
the  most  important  actions  in  an  IND  incident  response.  The  complications  of  having  limited 
communication  lines  with  local  responders,  uncertainty  in  where  the  damage  and  fallout  zones  are,  and 
the  scale  of  the  disaster  makes  collecting  more  information  for  improved  decision  making  a  challenge. 
Incident  commanders  must  therefore  prioritize  their  information  needs. 

Where  and  how  to  stage  incoming  resources?  (mentioned  by  7  of  24  interviewees) 

Once  outside  assets  become  available  in  the  days  after  the  disaster,  decisions  on  where  and  how  to 
stage  resources  for  maximum  efficacy  is  needed.  A  large  inflow  of  resources  can  cause  logistical 
challenges  if  staging  of  resources  is  not  well  planned. 

Another  major  challenge  is  identifying  the  extent  of  Once  outside  assets  become  available 

radiological  fallout  and  damage  to  infrastructure,  which  in  t/te  days  after  t/te  disaster,  decisions 

further  complicates  where  resources  can  be  staged  for  on  where  and  how  to  stage  resources 
integration  into  the  response.  Several  examples  were  for  maximum  efficacy  is  needed. 
provided  to  convey  the  importance  of  optimally  handling 
resource  staging. 
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The  first  example  is  from  Hurricane  Katrina  (2005).  The  commitment  of  resources  exceeded  the 
ability  for  emergency  managers  to  organize  and  use  them.  A  huge  backup  of  trucks  on  the  incoming  roads 
resulted,  and  it  became  very  difficult  for  the  emergency  managers  to  locate  the  specific  resources  and 
have  them  transported  to  the  areas  of  need. 

Another  example  was  provided  from  the  Haiti  Earthquake  (2010)  response.  Only  one  runway  was 
available  for  incoming  supplies.  As  a  result  of  aircraft  and  crew  logistics,  a  decision  needed  to  be  made 
whether  to  bring  in  a  SAR  team  or  a  medical/surgical  team,  the  team  not  chosen  would  not  arrive  for 
another  48  hours.  The  urban  SAR  team  was  brought  in  due  to  priorities  set  at  higher  levels  of  the  response 
coordination.  In  retrospect  the  SMEs  indicated  that  more  lives  would  probably  have  been  saved  if  the 
medical  team  arrived  first. 


Where  to  assemble  survivors?  (mentioned  by  7  of  24  interviewees) 

Once  people  have  been  evacuated,  where  will  they  be  assembled  for  medical  care  and  long-term 
sheltering  and  housing?  In  an  IND  incident  the  number  of  people  requiring  evacuation  can  be  daunting, 
and  the  resources  and  services  they  require  is  a  significant  challenge  to  provide.  It  will  be  difficult  to  find 
a  place  that  can  handle  a  large  amount  of  people.  SMEs  mentioned  that  there  may  be  push  back  from 
neighboring  areas  not  wanting  to  commit  to  receiving  large  amounts  of  survivors  for  an  undetermined 
period  of  time.  Fear  of  radiation  from  the  neighboring  areas  exacerbates  this  issue. 

How  to  recognize  as  a  rad/nuc  incident?  (mentioned  by  7  of  24  interviewees) 

Recognizing  the  incident  as  an  IND  detonation  is  challenging  due  to  the  difficulty  in  detecting  the 
presence  of  radiation.  Radiation  itself  cannot  be  identified  using  the  five  senses;  specialized  radiation 
monitoring  equipment  is  required  to  confirm  that  the  incident  involves  radiological  or  nuclear  material. 
SMEs  emphasized  that  many  localities  may  not  be  familiar  with  or  even  have  access  to  radiation 
monitoring  equipment,  or  that  equipment  may  be  not  calibrated  or  functional.  In  combination  with  the 
difficulties  in  communication,  obtaining  validated  information  that  radiation  is  present  may  be  difficult. 

There  are  visible  signs  that  the  incident  involved  an  IND  (e.g.,  the  intense  flash  of  light  and  dust 
cloud  [17]),  but  these  technical  characteristics  of  the  hazard  may  not  be  known  by  first  responders  or 
emergency  managers.  Consequently,  this  decision  may  be  difficult  for  some  types  of  responders,  but 
merely  a  matter  of  technical  training  for  others. 


Other  interviewees  mentioned  the  related  question  of  distinguishing  different  radiological 
incidents — most  notably  an  IND  vs.  a  radiological  dispersal  device  (RDD).  An  RDD  detonation  will  have 

a  much  smaller  explosion,  lower  radiation  intensity,  and  a 
smaller  fallout  zone  than  an  IND,  posing  little  short-term 
risk  to  first  responders  but  still  requiring  an  extensive 
cleanup  operation.  Responding  to  an  RDD  thus  requires  a 
slightly  different  set  of  priorities  than  either  an  IND  or  a 
non-radiological  incident.  Similarly,  nuclear  power  plant 
incidents  can  produce  intense  radiation  over  a  large  area, 


Recognizing  the  incident  as  an  IND 
detonation  is  challenging  due  to 
difficulties  in  detecting  the  presence 
of  radiation;  early  identification  is 
key  to  an  effective  response. 
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but  the  failsafe  precautions  in  place  generally  offer  a  much  longer  reaction  time  than  an  IND,  allowing 
time  to  evacuate  potentially  affected  populations  immediately  rather  than  sheltering  in  place. 

Where  are  the  damage  zones?  (mentioned  by  6  of  24  intennewees) 

Determining  the  boundaries  of  the  damage  zones  is  important  early  in  the  response  as  it  impacts 
where  lifesaving  and  other  response  operations  can  occur.  The  limited  amount  of  validated  information 
available  will  make  determining  these  zones  challenging.  Incorrectly  determining  the  zones  or  waiting  too 
long  to  determine  them  could  have  significant  impacts  on  the  number  of  lives  saved,  and  the  level  of  risk 
first  responders  are  exposed  to. 

Has  the  decision  making  staff  been  impacted?  (mentioned  by  6  of  24  interviewees) 

In  an  IND  detonation,  or  any  major  disaster  that  occurs  without  warning,  the  normal  decision 
making  staff  may  be  unavailable  because  they  themselves  are  impacted  by  the  incident.  The  impact  to  the 
decision-making  staff  could  be  either  direct  (e.g.,  they  were  within  the  blast  radius),  or  indirect  (e.g., 
roads  are  unpassable,  immediate  family  was  impacted,  or  communication  infrastructure  is  down).  Any 
impact  to  the  decision-making  staff  could  cause  confusion  in  the  direction  and  priorities  of  the  response 
and  the  chain  of  command,  which  would  lead  to  delays  in  the  response. 

The  SMEs  were  mostly  concerned  with  the  absence  of  the  higher-level  incident  commanders,  but 
they  were  also  concerned  with  the  impact  a  detonation  might  have  on  the  ranks  of  the  local  fire,  police, 
and  emergency  medical  service  members.  It  is  likely  that  at  least  some  would  be  impacted,  and  the  SMEs 
mentioned  that  the  organizations  must  react  and  self-correct  for  the  absence  of  the  standard  command 
structure. 

How  to  get  public  services  and  utilities  back  up?  ( mentioned  by  6  of  24  interviewees) 

A  few  SMEs  stressed  that  in  any  disaster  returning  functionality  to  public  resources  such  as  utilities, 
transportation,  and  businesses  is  the  most  important  part  of  response  and  recovery,  and  must  be  prioritized 
and  planned  for  from  the  beginning.  The  presence  of  radiation  and  the  impacts  on  infrastructure  will  add 
significant  challenges  to  the  restoration  of  services. 

How  and  where  to  decontaminate  people?  (mentioned  by  5  of  24  interviewees) 

Most  SMEs  noted  a  lack  of  public  education  and  awareness  about  the  effects  of  radiation.  Teaching 
people  how  to  properly  decontaminate  themselves  and  their  possessions  will  require  a  significant 
education  and  public  messaging  effort.  Keeping  track  of  where  people  and  vehicles  have  traveled  will 
make  determining  the  decontamination  requirements  difficult.  Surrounding  areas  to  the  disaster  will  be 
concerned  about  the  spread  of  radioactive  materials.  It  will  be  challenging  to  create  and  implement  a  plan 
to  handle  all  of  the  complexities  associated  with  decontaminating  such  a  large  number  of  people  in  a  short 
period  of  time  to  be  effective. 
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How  to  monitor  the  affected  population?  (mentioned  by  3  of  24  interviewees) 

Monitoring  the  movement  of  people  who  were  exposed  to  radiation  will  be  difficult,  and  significant 
challenges  will  exist  in  determining  who  has  received  doses  that  require  medical  attention  and  who  can 
self-decontaminate.  It  will  be  very  difficult  to  determine  the  radiation  dose  from  people’s  estimated 
locations  and  shelter  protection  factors.  Due  to  the  broad  lack  of  knowledge  of  radiation  by  the  public, 
medical  resources  may  become  overwhelmed  with  those  who  mistakenly  believe  they  require  care.  This 
decision  also  considers  how  to  limit  further  radiation  exposure  from  evacuating  populations. 

How  to  maintain  public  confidence  and  trust?  (mentioned  by  3  of  24  interviewees) 

Many  aspects  of  an  IND  incident  response  require  public  compliance  of  instructions  to  ensure  not 
only  their  own  safety  but  the  safety  of  others.  If  people  do  not  follow  directions  and  recommendations, 
they  could  delay  first  responders’  ability  to  help  others.  Maintaining  public  confidence  is  crucial  for 
maximum  lifesaving.  Emergency  managers  must  ensure  that  they  have  a  consistent  message,  and  that  it  is 
expressed  in  terms  that  can  be  interpreted  by  the  local  community,  and  that  the  message  reaches  the  right 
people. 

An  example  of  poor  communication  and  loss  of  public  confidence  and  cooperation  is  that  of  the  Elk 
River  chemical  spill  in  West  Virginia  in  2014.  The  state  of  West  Virginia  assumed  communication 
responsibilities  from  the  local  authorities,  and  used  messages 

based  on  advice  from  professionals  with  limited  knowledge  of  Maintaining  public  confidence, 
the  area.  According  to  one  SME,  the  “language  was  off’  which  leads  to  public  compliance 
which  led  to  suspicion  of  the  veracity  of  the  information  by  with  recommendations,  is  crucial 
the  local  community,  and  ultimately  confusion  and  a  lack  of  for  maximum  lifesaving. 
compliance. 

Should  first  responders  seal  off  areas  and  restrict  who  can  leave  and  enter?  (mentioned  by  3  of  24 

interviewees) 

The  complexities  of  radiation  raise  several  ethical  questions.  In  the  severe  and  moderate  damage  zones, 
which  also  have  dangerous  fallout  present,  there  will  likely  not  be  any  successful  lifesaving  operations 
[17].  Should  these  areas  be  restricted  to  responders  trying  to  help?  Should  the  public  be  restricted  from 
leaving  as  they  could  be  endangering  others  and  have  likely  received  a  fatal  dose? 

Is  this  a  single  detonation  or  is  there  a  secondary  device?  (mentioned  by  3  of  24  interviewees) 

Once  the  incident  has  been  established  as  an  IND  detonation,  the  next  question  SMEs  ask  is 
whether  it  is  a  single  incident  or  one  of  more  to  come.  If  more  detonations  are  probable,  then  all 
responders  should  be  kept  away  from  potentially  targeted  areas  to  conserve  resources.  However,  efforts  of 
responders  soon  after  a  detonation  could  save  more  lives.  Balancing  these  competing  interests  is 
demanding. 
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Note  that  this  item  was  not  frequently  mentioned  in  the  interviews  (mentioned  spontaneously  by  3 
of  24  interviewees),  but  it  was  consistently  given  a  high  rank  in  the  survey  (being  placed  in  the  top  half  in 
8  of  11  responses).  As  discussed  below,  we  believe  this  is  due  to  ambiguity  in  the  phrasing  of  the 
statement. 


Number  of  SMEs  That  Identified  the  Difficult  Decision 
(24  Total  Responses) 


How  to  allocate  finite  resources? 
How  to  phrase/disseminate  info  to  public  or  FRs? 
Whether  to  expose  first  responders  to  radiation? 

Where,  when,  and  how  to  evacuate? 
How  to  set  patient  triage  standards? 
Where  are  dangerous  fallout  and  hot  zones? 

Where  to  issue  shelter-in-place? 
How  to  develop  an  information  collection  plan? 
Where  to  stage  incoming  resources? 
Where  to  assemble  survivors? 
How  to  recognize  as  a  rad/nuc  incident? 
Where  are  the  damage  zones  (DZs)? 
Has  the  decision  making  staff  been  impacted? 
How  to  get  public  services  and  utilities  back  up? 
How  and  where  to  decontaminate  people? 
How  to  monitor  the  affected  population? 
How  to  maintain  public  confidence  and  trust? 
Do  FRs  restrict  who  can  leave/enter  areas? 
Single  detonation  or  is  there  a  secondary  device? 


Figure  2.  Difficult  decisions  emphasized  in  the  24  SME  interviews. 
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4.1.2  Decision  Analysis 


The  majority  of  the  difficult  decisions  identified  by  the  SMEs  are  required  in  some  form  for  other 
large-scale  incidents;  however  the  SMEs  highlighted  many  specific  challenges  and  complexities  of  a 
rad/nuc  incident  in  making  nearly  every  decision.  Their  emphasis  indicates  that  IND  incidents,  and  more 
generally  rad/nuc  emergencies,  do  present  distinct  challenges. 

Table  3  shows  a  timeline  of  an  IND  incident  response  planning  operations  from  the  IND  Response 
Planning  Tool  [11].  The  eight  most  common  difficult  decisions  as  identified  by  the  SMEs  are  highlighted 
in  red,  representing  decisions  mentioned  in  at  least  one  third  of  the  interviews.  There  is  not  a  direct  1 : 1 
mapping  from  our  list  to  the  response  planning  list,  but  all  of  the  eight  most  difficult  decisions  are 
represented  in  the  ten  items  highlighted  below  except  for  “developing  an  information  collection  plan,” 
which  touches  many  of  the  operations  in  the  0-6  hour  range.  Interestingly,  the  majority  of  the  difficult 
decisions  are  split  between  the  0-1  hour  and  1-6  hour  ranges,  and  all  within  24  hours  after  detonation. 
This  provides  evidence  that  a  driving  factor  in  the  difficulty  of  decisions  is  time  pressure. 


Table  3 

IND  Response  Planning  Timeline  [11] 


0-1  Hour 

1-6  Hours 

6-24  Hours 

24-48  Hours 

48-72  Hours 

•  Assess  Comm. 
Availability 

•  Characterize  the 
Weapon 

•  Determine  Down¬ 
wind  Direction 

•  Establish  Area 
Command 

•  Give  Shelter-in- 
Place  Command 

•  Inform  the  Public 

•  Notify 

Responders 

•  Recognize 

Incident  as  an 

IND 

•  Assess  EOC 

Integrity 

•  Assess  Utilities 

•  Characterize  the 
Incident 

•  Determine  and 
Monitor  Weather 

•  Determine 

Damage  and 

Fallout  Zones 

•  Determine  Medical 
Center  Conditions 

•  Establish  Dose 
Turnback  Levels 

•  Establish  Multiple 
Incident 

Commands 

•  Perform 

Operational  Risk 
Assessments 

•  Perform 
Radiological  Risk 
Assessments 

•  Plan  and  Allocate 
Resources 

•  Plan  and  Initialize 
Evacuation 

•  Preserve  Evidence 

•  Set  PPE  Standards 

•  Control  Access 

•  Control  Crowds 

•  Control  Fires 

•  Decontaminate 
People  and  Pets 

•  Initial  Mass 
Casualty  Triage 

•  Monitor  Population 

•  Perform  Sampling 
and  Laboratory 
Analysis 

•  Search  Damaged 
and  Collapsed 
Buildings 

•  Clear  Highways/ 
Freeways 

•  Establish  RTRS 
and  Other  Centers 

•  Integrate  Regional 
&  National  Assets 

•  Locate  and 

Extract  Survivors 

•  Distribute  Food 
and  Water 

Supplies 

•  Establish  Medical 
Aid  Stations 

•  Manage 

Fatal  ities/Casualti 

es 

•  Manage  Pets  and 
Service  Animals 

•  Recover 

Decedents 

•  Relocate 

Population 

•  Reunify  Families 

•  Disease 

Prevention  and 
Control  Measures 

25 


Table  4 


Number  of  Survey  Responses  Placing  Each  Decision 
in  the  Top  Half  of  the  Prioritized  List 


Decisions  (18  total) 

Survey  Responses 
Ranking  the  Decision 
in  Top  Half  (11  total) 

Where  to  issue  shelter-in-place  recommendation? 

10 

Where  are  the  damage  zones? 

10 

Where  are  the  dangerous  fallout  and  hot  zones? 

10 

Where,  when,  and  how  to  evacuate? 

9 

Whether  to  expose  first  responders  to  radiation  and  to  what  extent? 

8 

Is  this  a  single  detonation  or  is  there  a  secondary  device? 

8 

How  to  allocate  finite  resources? 

6 

How  to  recognize  as  a  rad/nuc  incident? 

6 

Should  first  responders  seal  off  areas  and  restrict  who  can  leave  and  enter? 

6 

Where  to  assemble  survivors? 

5 

How  to  select/phrase  info  disseminated  to  the  public  and  first  responders? 

4 

How  to  maintain  public  confidence  and  trust? 

4 

How  to  set  patient  triage  standards? 

3 

Has  the  decision-making  staff  been  impacted? 

3 

How  and  where  to  decontaminate  people? 

3 

Where  and  how  to  stage  incoming  resources? 

2 

How  to  develop  an  information  collection  plan? 

2 

How  to  get  public  services  and  utilities  back  up? 

0 

The  survey  results  largely  matched  the  interviews — the  top  eight  most  mentioned  decisions  in  the 
interviews  (decisions  mentioned  by  at  least  one  third  of  interviewees)  were  also  ranked  in  the  top  half  of 
most  important  decisions  by  the  majority  of  the  survey  responses,  with  the  following  exceptions: 

•  Determining  if  there  was  a  single  detonation  or  if  there  is  a  secondary  device.  Only  3  of  24 
interviewees  spontaneously  mentioned  this  topic,  but  when  presented  with  the  issue  in  a  ranked 
list  it  was  rated  much  higher;  8  of  1 1  responses  placed  it  in  the  top  half  of  their  importance  list. 
The  likely  explanation  is  that  the  presence  of  a  secondary  device  would  have  a  large  impact  on 
the  scale  of  the  incident,  but  that  the  risk  of  a  second  incident  has  little  impact  on  how  a 
response  is  managed.  Thus,  responders  may  rate  it  high  in  terms  of  impact  on  the  population,  or 
low  in  terms  of  impact  on  the  response  effort.  We  interpret  this  to  be  a  case  where  a  critical 
issue  was  simply  overlooked  during  the  interview  process.  A  few  SMEs  did  specifically 
mention  that  the  risk  of  a  second  incident  might  justify  holding  back  some  resources  at  the 
federal  level,  thereby  affecting  the  response,  but  the  general  consensus  among  the  SMEs 
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seemed  to  be  that  the  hard  decisions  concerning  a  second  device  would  be  for  law  enforcement 
not  emergency  response. 

•  The  importance  of  phrasing  and  disseminating  information  to  the  public  was  2nd  based  on 
interview  responses,  but  only  4  of  1 1  survey  responses  put  it  in  the  top  half  of  their  ranked 
decisions.  This  discrepancy  may  reflect  an  uncertainty  of  authority.  The  interview  questions 
focused  on  what  the  most  important  decisions  were,  while  the  survey  asked  what  decisions 
were  most  important  to  the  responder’s  own  role.  Our  interpretation  is  that  the  decision  of  what 
to  tell  the  public  is  critical  overall,  but  most  responders  do  not  think  it  is  part  of  their 
responsibilities. 

•  Deciding  how  to  set  patient  triage  standards  was  mentioned  by  10  of  24  interviewees,  but 
only  3  of  1 1  survey  responses  ranked  it  in  the  top  half.  Our  interpretation  is  that  this  decision  is 
important  and  difficult,  but  not  an  immediate  priority.  There  is  time  to  set  triage  standards  so, 
while  it  will  be  morally  and  strategically  difficult,  it  does  not  face  the  same  time  pressure  as 
other  decisions  on  the  list.  The  ranked  list  may  have  encouraged  the  responders  to  prioritize 
items  by  timeliness,  while  the  interviews  encouraged  prioritization  by  difficulty.  Those  two 
criteria  often  align,  since  most  of  the  casualties  of  a  large-scale  disaster  occur  early  in  the 
incident,  which  explains  why  this  discrepancy  did  not  affect  more  items  on  the  list. 

Note  that  only  18  decisions  were  included  in  the  survey.  The  19th  item,  “How  to  monitor  the 
affected  population,”  was  identified  in  interviews  after  the  survey  had  been  distributed  to  a  portion  of 
SMEs.  We  chose  to  keep  it  out  of  the  remaining  surveys  for  consistency. 

4.2  SKILLS  IDENTIFIED  AS  CRITICAL  TO  DECISION  MAKING  DURING  EMERGENCY 

RESPONSE 

Table  5  lists  all  skills  emphasized  by  two  or  more  SMEs  as  being  critical  during  the  response  to  an 
IND  detonation  or  other  large-scale  disaster.  Skills  mentioned  by  two  or  less  SMEs  were  omitted  as 
nonrepresentative.  The  rest  of  this  section  provides  a  more  detailed  definition  and  explanation  for  each 
skill  mentioned,  accompanied  by  examples  when  provided  by  the  SMEs  for  clarification.  Similar  to  the 
explanations  for  the  key  decisions,  the  level  of  detail  of  the  skills  given  below  is  reflective  of  the  detail 
provided  by  the  SMEs  and  the  number  of  SMEs  that  discussed  the  skills.  Our  interpretation  is  that  skills 
emphasized  by  more  SMEs  and  discussed  in  greater  detail  are  likely  to  be  the  more  important  skills,  or 
skills  that  the  SMEs  are  concerned  will  be  lacking  during  an  incident.  The  section  ends  with  a  detailed 
breakdown  and  classification  of  the  skills  (Figure  3). 

Note  that  the  low  mention  rate  of  a  skill  does  not  mean  that  the  skill  is  not  important.  A  low  rate  of 
mention  might  simply  indicate  one  of  the  following: 

•  The  SMEs  believe  that  everyone  under  consideration  for  those  roles  has  that  skill  (i.e.,  it  is  not 
a  distinguishing  characteristic  of  the  best  emergency-management  decision  makers). 
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•  The  skill  is  easily  taught  or  measured  for  currently;  or  it  is  not  a  critical  skill  for  a  specific  type 
of  emergency  manager  (i.e.,  it  is  not  of  high  importance  to  FEMA  IMAT  emergency  managers, 
but  it  may  be  critical  to  a  first  responder). 

•  The  skill  may  still  need  to  be  prioritized  through  conventional  preparation  techniques  and 
included  in  evaluations. 

We  interpret  a  low  mention  rate  to  mean  that  these  skills  do  not  need  to  be  a  focus  of  augmentation  of 
training  and  evaluation,  and  thus  they  are  of  lesser  interest  to  this  project’s  goals. 

Table  5 

List  of  Skills  Emphasized  by  Interviewed  SMEs 


Skills  Emphasized  Page  Number 

Make  a  timely  decision  with  limited  information  29 

Determine  reliable  information  and  sources  29 

Have  relevant  technical  domain  knowledge  for  your  position  30 

Leadership  30 

Communication  30 

Ability  to  adapt  plans  and  improvise  solutions  based  on  changing  conditions  31 

Apply  resources  effectively  and  efficiently  31 

Synthesize  information  quickly,  gain  situational  awareness  32 

Experience  working  with  the  response  team  or  network  32 

Selective  attention  32 

Cost/benefit  analysis  32 

Forward  thinking,  able  to  adjust  decisions  33 

Teamwork  33 

Focus  under  stress,  detach  from  emotional  aspects  33 

Courage  (confidence  to  make  a  decision)  33 

Provide  vision  of  outcome,  set  priorities  34 

Compassion  34 

Capability  to  influence  and  collaborate  34 

Experience  with  strategic  decisions  35 

Understand  and  navigate  the  political  nature  of  the  incident  35 
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4.2.1  Skill  Descriptions  and  Examples 

Make  a  timely  decision  with  limited  information  (mentioned  by  15  of  24  interviewees) 

The  most  mentioned  critical  skill  is  being  able  to  make  a  timely  decision  with  limited  validated 
information.  Many  SMEs  expressed  that  during  a  large  scale  incident  there  are  competing  objectives  of 
spending  time  and  effort  to  gain  validated  information  to  make  the  best  decision,  or  making  a  quick 
decision  based  on  the  information  at  hand.  The  SMEs  said  that  the  most  experienced  and  successful 
emergency  managers  know  how  to  balance  these  two  objectives  to  make  a  decision  that  is  “good  enough” 
to  keep  the  response  moving  in  the  right  direction. 

The  interviewed  SMEs  made  the  following  comments  on  this  topic: 

•  Make  a  decision  and  move  on;  someone  who  is  unable  or  hesitant  will  be  disregarded. 

•  There  is  no  time  for  “checklisting”  to  get  all  the  details  right. 

•  There  is  no  ultimate  right  and  wrong,  but  actions  must  be  taken. 

•  Use  an  abbreviated  analytical  approach,  put  resources  into  action. 

•  Take  in  information,  then  gather  and  deploy  resources. 

•  It  is  unlikely  you  will  receive  full  information;  making  a  decision  when  needed  is  more 
important. 

•  Make  decisions  quickly  with  limited  validated  information.  As  more  information  is  available, 
the  quality  of  decisions  should  improve. 

•  Create  solutions  that  move  in  the  right  direction,  the  solution  does  not  need  to  be  optimal. 
Determine  reliable  information  and  sources  (mentioned  by  11  of  24  interviewees) 

As  noted  in  other  skills  and  decisions,  ample  validated  information  required  for  optimal  decision¬ 
making  is  often  limited.  Consequently,  the  ability  to  determine  what  data  and  data  sources  are  reliable  is  a 
crucial  decision  making  skill  in  an  IND  incident  response. 

A  successful  response  to  a  large  scale  disaster  like  an  IND  detonation  will  require  a 
multidisciplinary  response.  For  emergency  managers  to  be  successful  they  must  be  able  to  rely  on  other 
SMEs  to  provide  information  and  make  decisions.  For  this  network  of  SMEs  to  be  effective,  several 
necessary  conditions  were  mentioned: 

•  The  SMEs  must  be  in  place  and  available  on  short  notice. 

•  The  decision  makers  must  know  that  SMEs  exist,  how  to  access  them,  and  when  it  will  be 
helpful  to  do  so. 
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•  The  decision  makers  need  to  act  on  the  knowledge  they  receive,  trusting  the  SMEs’  knowledge 
without  over-relying  on  the  accuracy  of  data  collected  from  the  field. 

Have  relevant  technical  domain  knowledge  for  your  position  (mentioned  by  11  of  24  interviewees) 

This  skill  refers  to  being  familiar  with  the  emergency  response  plan,  and  any  technical  knowledge 
that  may  be  required  to  make  informed  decisions  (such  as  impact  of  radiation,  best  practices  for  selecting 
evacuation  routes,  or  expected  impacts  on  infrastructure).  This  skill  was  most  emphasized  by  radiological 
health  SMEs.  Decision  makers  do  not  need  extensive  knowledge  on  the  radiation  hazard  but  should  be 
familiar  with  the  types  of  data  that  will  aid  their  decision-making.  These  are  the  skills  that  are  generally 
accepted  as  being  trainable  via  classroom  teaching  or  reading  of  the  literature,  although  they  may  be 
further  reinforced  through  exercises  and  on-the-job  experience. 

Leadership  (mentioned  by  11  of  24  interviewees) 

Leadership  in  the  emergency  response  context  is  described  by  the  SMEs  as  being  able  to  set 
priorities,  assign  responsibilities,  and  make  sure  the  team  has  the  proper  guidance  to  be  effective  in  their 
tasks.  Below  is  a  list  of  quotes  used  by  SMEs  when  describing  leadership  qualities: 

•  Delegate  authority  and  responsibilities 

•  Make  sure  all  people  are  being  used  effectively 

•  Ability  to  communicate  intent 

•  Provide  purpose,  motivation,  direction,  and  insight  for  workers  to  meet  goals 

•  Provide  structure  to  disparate  elements 

•  Set  priorities 

•  Understand  talent  and  align  for  best  performance 

•  Make  sure  team  members  are  working  on  the  right  tasks 

•  Leaders  should  not  be  the  ones  doing  the  work 

Communication  (mentioned  by  10  of  24  interviewees) 

Accurately  transferring  information  between  individuals  is  essential  for  any  job  involving  multiple 
people  to  be  carried  out  efficiently.  When  time  pressure  and  limited  resources  are  added,  the  ability  to 
articulate  information  in  a  way  that  others  will  comprehend  becomes  imperative  for  the  response  to 
progress  quickly. 

One  example  of  communication  issues  comes  from  the  Fukushima  Daiichi  nuclear  disaster  (2011). 
The  local  medical  health  professional  on  staff  at  the  U.S.  embassy  in  Tokyo  was  not  effectively  advising 
Americans  in  the  area  about  health  concerns  and  promoting  confidence  in  the  declared  exclusion  area. 
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Consequently,  a  different  U.S.  expert  was  brought  in  to  take  over  the  role;  this  person  provided  reliable 
information  that  was  trusted  by  the  U.S.  citizens. 

Communication  includes  both  clarity  and  appropriate  filtering.  Several  local  emergency  managers 
and  rad/nuc  specialists  mentioned  the  importance  of  exercising  judgment  about  when  to  pass  information 
up  the  chain  of  command  and  when  to  avoid  distracting  that  layer  of  decision  makers  with  an  extra  level 
of  detailed  information.  Similar  concerns  were  expressed  about  when  to  pass  information  down  to 
responders  in  the  field,  and  when  to  withhold  unnecessary  or  potentially  misleading  detail. 

Ability  to  adapt  plans  and  improvise  solutions  based  on  changing  conditions  (mentioned  by  10  of  24 

interviewees) 

Knowing  the  plan  and  the  proper  courses  of  action  is  necessary  for  any  responder  to  make  the 
correct  decision.  However,  it  is  just  as  important  to  be  able  to  adapt  the  plan  and  improvise  a  new  solution 
to  fit  the  current  situation,  and  address  incident-specific  complications.  Knowing  the  plan  is  the  first  step 
towards  making  the  correct  decision,  but  understanding  the  intent  of  the  plan  will  help  the  responder 
make  the  best  decisions  given  the  incident  specifics  (which  is  likely  to  differ  from  the  situation  assumed 
when  the  plan  was  created). 

An  example  of  plan  improvisation  mentioned  by  two  SMEs  was  the  implementation  of  the  Shelter 
and  Temporary  Emergency  Power  (STEP)  program  after  Hurricane  Sandy  (2012)  in  New  York  City 
(NYC).  An  initial  response  objective  following  a  major  hurricane  where  homes  are  damaged  and 
populations  are  displaced  is  to  provide  temporary  shelter.  However,  space  and  cost  restrictions  in  NYC 
made  the  typical  temporary  sheltering  prohibitive.  Instead,  a  decision  was  made  to  have  local  workers 
quickly  fix  homes  and  return  them  to  a  livable  state,  so  that  citizens  could  return  home  and  plan  for  more 
comprehensive  repairs. 

Another  example  from  Hurricane  Sandy  was  altering  the  community  relations  and  communication 
methods.  In  other  disaster  responses,  communication  was  accomplished  using  electronic  means;  however, 
because  electricity  remained  out  for  many  people  for  longer  than  expected,  protocols  were  adapted  to 
send  out  first  responders  to  directly  communicate  with  people  in  those  areas.  The  same  goals  were 
achieved  by  a  different  implementation. 

Apply  resources  effectively  and  efficiently  (mentioned  by  8  of  24  interviewees) 

This  skill  is  very  similar  to  the  difficult  decision  of  allocating  finite  resources.  Effectively  and 
efficiently  applying  resources,  whether  they  are  supplies  or  personnel,  is  a  vital  skill  since  a  large  part  of 
the  emergency  managers’  responsibilities  are  to  coordinate  asset  integration  from  neighboring  or  higher 
levels  of  government  into  the  response.  SMEs  emphasized  that  the  most  successful  responders  focus  more 
on  applying  the  resources  to  be  effective  and  make  progress,  i.e.,  “put  resources  to  use,”  rather  than 
spending  additional  time  trying  to  maximize  nuances. 
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Synthesize  information  quickly,  gain  situational  awareness  (mentioned  by  8  of  24  interviewees) 

This  skill  is  the  ability  to  rapidly  gain  and  synthesize  information  to  build  an  understanding  of  the 
current  state  of  the  response,  often  referred  to  by  the  SMEs  as  having  good  situational  awareness.  Quickly 
creating  an  understanding  of  the  incident  and  response  operations  allows  an  emergency  responder  to 
accurately  determine  priorities  and  begin  taking  actions  to  direct  the  response  in  the  right  direction. 
Without  having  a  sound  understanding  of  the  situation  on  the  ground,  it  is  difficult  to  allocate  resources 
and  direct  response  operations. 

Experience  working  with  the  response  team  or  network  (mentioned  by  8  of  24  interx’iewees) 

Many  SMEs  expressed  that  team  cohesion  is  a  necessity  for  an  efficient  response.  Prior  experience 
working  together  gives  team  members  an  opportunity  to  build  trust  in  one  another  and  learn  the  strengths 
and  weaknesses  of  each  team  member.  This  way  each  other’s  actions  can  be  anticipated  and  the  response 
is  quick  and  efficient;  during  an  incident  there  is  no  time  luxury  to  build  these  relationships.  One  SME 
described  the  importance  of  team  experience  by  stating:  “team  cohesion  is  more  important  than  having 
the  best  and  the  brightest.” 

State  level  emergency  managers  spoke  about  this  as  having  experience  working  with  the  network  of 
other  agency  representatives  that  will  participate  in  an  incident  response.  They  described  that  if 
relationships  already  existed  there  would  be  less  time  trying  to  determine  if  a  person  could  provide 
reliable  information,  or  if  a  person  could  be  trusted  to  make  decisions  on  their  own.  Rad/nuc  domain 
SMEs  spoke  of  this  experience  as  building  trust  and  familiarity  with  those  taking  radiation  measurements 
and  the  equipment  being  used.  Many  SMEs  spoke  of  building  this  experience  during  multi-organization 
exercises. 

Selective  attention  (mentioned  by  7  of  24  interviewees) 

Selective  attention  is  the  ability  to  focus  on  what  is  important  in  the  presence  of  distractions.  SMEs 
noted  this  skill  as  useful  not  just  for  sorting  through  information  but  also  for  prioritizing  actions.  The 
ability  to  hone  in  on  helpful  and  useful  bits  of  information,  and  not  be  distracted  by  the  sheer  amount  of 
data  separates  the  experienced  from  the  inexperienced  personnel.  An  important  use  of  this  skill  is 
determining  which  decisions  are  most  important  and  require  immediate  attention  and  which  can  wait.  It  is 
often  easier  to  answer  the  simple  questions  and  put  the  more  complicated  issues  to  the  side;  however 
determining  which  questions  are  important  to  answer  first  and  taking  care  of  them  accordingly  will  lead  to 
a  more  effective  response. 

Cost/benefit  analysis  (mentioned  by  7  of  24  interviewees) 

Many  of  the  difficult  decisions  made  in  an  IND  incident  response  require  balancing  competing 
objectives,  such  as  whether  to:  wait  or  send  in  responders,  evacuate  or  she  Iter- in-place,  treat  all  injured 
people  or  only  those  who  can  survive.  Emergency  managers  must  be  able  to  weigh  the  risks  and  rewards 
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of  taking  or  delaying  a  certain  action  when  there  are  many  competing  priorities.  One  SME  stated  that 
these  trade-offs  do  not  need  to  be  made  during  more  routine  disasters  when  resources  are  not  over-taxed. 
In  an  IND  incident  response  where  resources  become  extremely  limited,  consideration  of  the  risk  and 
reward  for  actions  will  be  essential  especially  when  considering  lifesaving  operations. 

Forward  thinking,  able  to  adjust  decisions  (mentioned  by  7  of  24  interviewees) 

A  large  scale  disaster  like  an  IND  detonation  must  be  treated  as  a  fluid  incident.  The  response  will 
change  as  time  progresses  and  more  information  becomes  available.  Decisions  must  be  made  to  consider 
not  only  the  current  state  of  events  but  also  the  possible  progressions  that  could  occur.  As  the  response 
advances,  responders  must  be  able  to  change  or  alter  decisions  that  were  made  with  outdated  information 
and  not  get  stuck  following  an  inadequate  plan.  An  understanding  of  the  “big  picture”  and  potential 
“global  impact”  is  required  to  predict  any  potential  consequences  that  may  result  from  any  particular 
decision.  Unintended  consequences  can  be  technical  (interconnected  critical  infrastructure),  social  (how 
the  population  responds  to  the  situation),  political  (how  leaders  and  media  react),  or  a  complex 
combination. 

Teamwork  (mentioned  by  7  of  24  interviewees) 

The  ability  to  work  on  a  team,  including  being  able  to  rapidly  generate  team  cohesion  and  work  at  a 
level  of  high  efficiency  with  others,  is  an  important  skill  identified  by  SMEs.  Above  in  the  experience 
with  response  team  skill,  several  benefits  are  revealed  from  having  prior  work  experience  with  team 
members.  In  contrast,  teamwork  covers  cohesion  with  collaborators  whether  or  not  you  have  prior 
experience  working  with  them. 

Focus  under  stress,  detach  from  emotional  aspects  (mentioned  by  6  of  24  interviewees) 

In  any  disaster  where  there  is  the  potential  for  a  significant  loss  of  life,  injury,  and  widespread 
damage  to  personal  property,  emotional  reactions  and  stress  from  those  responding  are  expected, 
especially  by  those  who  may  have  to  make  decisions  and  take  actions  that  could  directly  impact  the  lives 
of  others.  An  important  skill  of  an  effective  emergency  manager  is  to  be  able  to  detach  from  the 
emotional  aspects  of  the  decision,  remain  calm  and  focused,  and  make  the  best  decision  they  can.  This 
skill  was  emphasized  as  especially  important  for  an  IND  incident  response  because  first  responders  may 
encounter  individuals  who  may  appear  to  be  rescuable  but  have  sustained  high  levels  of  exposure  to 
radiation.  The  sheer  scale  of  suffering  from  an  IND  detonation  might  overwhelm  responders  who  are 
normally  calm  under  pressure,  and  will  increase  the  number  of  emergency  managers  with  personal 
connections  to  the  affected  public. 

Courage  (confidence  to  make  a  decision)  (mentioned  by  5  of  24  interviewees) 

Courage  was  described  as  having  the  confidence  to  make  a  decision  to  do  what  is  right  when  an 
emergency  manager  may  be  unsure  of  his/her  authority  or  ability  to  make  such  a  choice.  This  skill  is 
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closely  related  to  many  of  the  thoughts  shared  concerning  making  a  timely  decision  with  limited 
information.  SMEs  described  that  gathering  the  required  approvals  to  make  sure  they  have  the  authority  to 
take  a  given  action  may  delay  the  decision  or  action.  The  following  are  quotes  from  SMEs  describing  this 
skill:  “do  what  you  know  is  right  and  worry  about  the  repercussions  later”  and  “work  like  you  don’t  care 
if  you  have  a  job.” 

An  example  was  shared  from  Hurricane  Sandy  from  the  New  Jersey  shore.  There  was  an  excessive 
amount  of  debris  impeding  the  progress  of  the  response.  Generally,  there  are  regulations  concerning  who 
can  move  debris,  where  it  can  be  moved,  what  contents  could  be  dangerous,  etc.  A  decision  was  made  to 
temporarily  postpone  investigating  the  debris  movement  regulations  to  make  sure  that  the  response  could 
progress.  This  decision  was  later  perceived  as  beneficial. 

Provide  vision  of  outcome,  set  priorities  (mentioned  by  5  of  24  interviewees) 

The  ability  to  provide  a  vision  of  desired  outcomes  and  to  set  priorities  to  achieve  those  outcomes 
was  highlighted  by  the  SMEs.  The  magnitude  of  a  large  incident  can  prohibit  emergency  managers  from 
having  the  capacity  to  make  the  decisions  on  response  implementation  details.  Their  focus  is  to  provide 
clear  direction  of  the  response  priorities  and  goals  that  will  achieve  the  desired  outcome,  not  the  methods 
used  to  get  there. 

Note  that  this  item  was  not  frequently  mentioned  in  the  interviews  (mentioned  spontaneously  by  5 
of  24  interviewees),  but  it  was  generally  given  a  high  rank  in  the  survey  (being  placed  in  the  top  half  in  7 
of  1 1  responses).  So,  while  the  issue  was  not  on  the  minds  of  many  SMEs  when  interviewed,  when 
prompted  with  the  skill,  it  was  considered  to  be  relevant. 

Compassion  (mentioned  by  4  of  24  interviewees) 

For  responders  and  emergency  managers  to  be  successful  they  need  to  have  a  genuine  concern 
about  those  they  are  trying  to  help.  One  of  the  National  IMAT  leaders  mentioned  that  two  of  the  three 
primary  goals  in  emergency  response  are  to  “save  lives”  and  “reduce  suffering.”  Compassion  is  also 
required  for  one’s  colleagues.  One  SME  stated,  “the  disaster  creates  all  the  drama  we  need”;  therefore, 
animosity  towards  others  is  not  going  to  help  the  response. 

Capability  to  influence  and  collaborate  (mentioned  by  3  of  24  inten’iewees) 

This  skill  specifically  refers  to  influencing  and  collaborating  across  organization  boundaries,  not 
just  within  a  team  or  along  normal  lines  of  authority.  FEMA’s  role  through  the  IMAT  is  to  support  the 
state  and  local  governments  in  their  responses.  We  heard  concerns  that  state  and  local  incident 
commanders  may  be  reluctant  to  accept  federal  aid,  sometimes  assuming  the  attitude  of  “we’ve  got  this” 
since  federal  aid  may  not  be  available  for  48-72  hours.  In  an  IND  detonation,  state  and  local  resources 
will  be  over-taxed,  making  it  especially  important  to  draw  on  appropriate  federal  support.  Several  SMEs 
stated  that  in  order  to  overcome  those  potential  obstacles,  the  federal  side  must  be  able  to  influence  and 
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collaborate  with  state  and  local  incident  commands  so  that  they  make  the  correct  decisions  without 
feeling  like  they  are  losing  control  of  the  incident.  State  and  local  SMEs  alluded  to  similar  issues  in 
regards  to  collaborating  with  neighboring  states  or  counties. 

Local  and  state  officials  may  be  hesitant  to  accept  federal  support  for  a  range  of  reasons,  but  this 
can  be  overcome  by  incrementally  building  trust.  Accomplish  small  tasks  first,  even  if  they  are  not  the 
highest  priority  issues,  so  that  when  the  bigger  tasks  appear  you  already  have  credibility.  SMEs  described 
this  as  “leading  through  actions  and  success  and  not  just  talk.”  This  approach  can  feel  backwards,  since  it 
may  involve  staying  out  of  critical  decisions  early  in  the  response,  but  it  will  be  for  the  greater  good 
throughout  the  incident.  It  is  best  if  such  trust  is  acquired  before  the  incident  through  other  small  scale 
interactions,  but  SMEs  noted  that  it  can  still  be  built  on  the  spot. 

Experience  with  strategic  decisions  (mentioned  by  3  of  24  interviewees) 

Strategic  decision-making  involves  making  decisions  based  on  long-term  and  large-scale  priorities 
and  objectives;  recognizing  that  a  decision  could  have  a  significant  impact  on  the  “big  picture”  and 
making  it  in  that  context  is  imperative  for  an  effective  response.  Having  been  involved  with  making 
strategic  decisions  in  past  incidents  is  currently  one  of  the  only  ways  to  be  confident  that  a  person  will  be 
competent  when  it  comes  to  making  them  in  the  future.  The  details  of  this  skill  are  somewhat  vague, 
reflecting  a  low  level  of  detail  and  emphasis  placed  on  it  by  SMEs  during  the  interviews. 

Understand  and  navigate  the  political  nature  of  the  incident  (mentioned  by  3  of  24  interviewees) 

In  any  disaster  there  will  be  interactions  with  political  leaders.  It  is  important  for  those  making 
decisions  for  the  emergency  response  to  be  cognizant  of  the  priorities  of  public  officials.  Since  it  is 
imperative  for  all  parties  to  be  working  towards  the  same  goals  for  the  maximum  effectiveness, 
emergency  managers  need  to  navigate  political  pressures  and  understand  the  additional  goals  held  by 
leaders,  such  as  gamering  public  support  for  an  action,  presenting  a  positive  view  of  the  response  effort, 
and  responding  to  issues  capturing  media  attention. 

For  the  majority  of  critical  decisions,  the  role  of  a  FEMA  representative  during  an  IND  incident  is 
to  advise,  not  decide.  Therefore,  the  ability  to  recognize,  navigate,  and  influence  the  social,  political,  and 
organizational  structures  present  in  the  response  effort  is  a  critical  skill.  Without  that  skill,  there  will  be 
no  opportunity  to  bring  the  other  skills  to  bear  and  to  contribute  to  the  critical  decisions.  One  SME 
phrased  the  sentiment  “If  they  don’t  smile  when  we  walk  in  the  room,  then  we’ve  already  failed.” 
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Number  of  SMEs  That  Identified  a  Skill  (24  Total  Responses) 


Make  a  timely  decision  with  limited  information 
Determine  and  use  reliable  information/sources 
Adapt  plans,  improvize  solutions 
Apply  resources  effectively  /  efficiently 
Synthesize  information,  gain  situational  awareness 

Selective  attention 
Cost  /  benefit  analysis 
Forward  thinking,  able  to  adjust  decisions 
Focus  under  stress,  detach  from  emotional  aspects 
Confidence,  courage  to  make  a  decision 
Provide  vision  of  outcome  /  set  priorities 

Leadership 

Communication 

Teamwork 

Compassion 

Capability  to  influence  and  collaborate 
Experience  working  with  the  response  team  or  network 

Experience  with  strategic  decisions 

Understand  and  navigate  the  political  nature  of  the 
incident 

Have  relevant  technical  domain  knowledge 


Figure  3.  Required  skills  emphasized  in  the  24  SME  interviews. 
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4.2.2  Skill  Analysis 


The  skills  emphasized  by  the  SMEs  have  been  organized  into  four  categories: 

1 .  Cognitive  Skills  are  mental  abilities  that  aid  in  performing  data  processing  tasks 

2.  Interpersonal  Skills  refers  to  skills  a  person  uses  to  interact  with  others 

3.  Experience  is  having  prior  opportunities  to  practice  or  directly  encounter  a  situation 

4.  Domain  Knowledge  refers  to  information  retention  relevant  to  a  given  subject 

Figure  4  shows  the  relative  proportion  of  reports  of  skills  according  to  these  four  categories.  There  were 
149  total  skill  reports  (the  sum  of  the  counts  in  Figure  3).  The  majority  of  skill  reports  were  for  skills 
classified  as  cognitive  (89  reports),  about  a  quarter  of  reports  were  for  skills  classified  as  interpersonal  (35 
reports),  and  the  remainder  were  classified  as  either  experience  based  (14  reports)  or  domain  knowledge 
(11  reports).  This  breakdown  suggests  that  extensions  to  current  training  and  preparation  methods  should 
primarily  focus  on  targeting  cognitive  and  interpersonal  skills.  Note  that  this  does  not  mean  that 
knowledge  and  experience-based  skills  are  not  important  (indeed,  there  is  no  doubt  that  they  are 
essential),  merely  that  they  are  already  well  addressed  by  current  training  and  preparation,  and  thus  were 
not  emphasized  as  an  area  of  concern.  Also  note  that  both  of  those  categories  contained  skills  that 
individually  rated  high  on  the  list,  indicating  value  in  better  addressing  all  four  areas  in  training. 


Domain 

Knowledge 

«°/~  — 


Experience 
10%  A 


Figure  4.  Breakdown  of  skill  reports  by  type  of  skill. 
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Table  6 


Number  of  Survey  Responses  Placing  Each  Skill  in 
the  Top  Half  of  the  Prioritized  List 


Survey  Responses 

Skills  (21  total) 

Ranking  the  Skill  in 

Top  Half  (11  total) 

Synthesize  information  quickly,  gain  situational  awareness 

11 

Make  a  timely  decision  with  limited  information 

10 

Apply  finite  resources  effectively  and  efficiently 

8 

Ability  to  adapt  plans  and  improvise  solutions  based  on  changing  conditions 

7 

Focus  under  stress,  detach  from  emotional  aspects 

7 

Provide  vision  of  outcome,  set  priorities 

7 

Leadership  (inspiring  and  guiding  a  team  to  be  effective) 

7 

Selective  attention  (focusing  on  critical  information  and  decisions) 

6 

Communication  (intelligently  sharing  and  interpreting  shared  information) 

6 

Have  relevant  technical  domain  knowledge  for  your  position 

6 

Teamwork  (working  as  part  of  a  larger  team) 

5 

Capability  to  influence  and  collaborate 

5 

Experience  with  making  strategic  decisions 

5 

First  responder  and  emergency  response  experience 

5 

Determine  reliable  information  and  sources 

4 

Forward  thinking,  able  to  adjust  decisions 

4 

Compassion  (empathy  for  those  who  are  affected  or  responding) 

3 

Experience  working  with  the  response  team  or  network 

3 

Understand  and  navigate  the  political  nature  of  the  incident 

1 

Cost/benefit  analysis,  both  of  actions  taken  and  not  taken 

0 

Understand  NIMS/ICS 

0 
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The  survey  results  largely  matched  the  interviews — the  top  eight  most  mentioned  skills  in  the 
interviews  (skills  mentioned  by  at  least  one  third  of  interviewees)  were  also  ranked  in  the  top  half  of 
importance  by  a  majority  of  the  survey  responses,  with  the  following  exceptions: 

•  The  skill  of  providing  vision  of  the  outcome  and  setting  priorities  rated  much  higher  in  the 
survey  than  in  the  interviews.  Only  5  of  24  interviewees  spontaneously  mentioned  this  skill,  but 
when  presented  with  the  issue  in  a  ranked  list,  7  of  1 1  responses  placed  it  in  the  top  half.  We 
attribute  this  discrepancy  to  the  similarly  in  this  skill  and  the  “leadership”  and  “communication” 
skills,  which  scored  highly  in  both  the  interviews  and  the  survey. 

•  The  benefit  of  experience  working  with  the  response  team  or  network  was  emphasized  in  8  of 
24  interviews,  but  was  put  in  the  top  half  of  the  survey  results  in  only  3  of  1 1  responses.  That  skill 
appears  to  be  valued  and  salient  but  not  considered  to  be  as  critical  as  others. 

•  The  skill  of  determining  reliable  information  sources  was  mentioned  in  1 1  of  24  interviews, 
but  only  4  of  1 1  surveys  put  it  in  the  top  half  of  important  skills.  We  suspect  that  this  discrepancy 
is  due  to  a  lack  of  clarity  in  the  phrasing  of  the  survey.  When  we  tallied  interview  results,  we 
considered  discussion  of  knowing  how  to  access  SMEs  to  be  an  instance  of  determining  reliable 
information  sources,  but  anecdotal  queries  lead  us  to  believe  that  survey  responders  may  have 
interpreted  it  more  narrowly  to  just  include  interpreting  data  from  sensors  and  field  reports.  While 
both  are  relevant,  the  broader  interpretation  of  the  category  naturally  ranks  higher. 

Note  that  the  skills  “understand  NIMS/ICS”  and  “first  responder  and  emergency  response 
experience”  were  left  off  of  the  skill  descriptions  above  because  they  were  only  emphasized  by  two  SMEs 
during  the  interviews;  however  they  were  included  in  the  survey  so  we  have  shown  them  in  the  results 
above.  Also  the  skill,  “courage  (confidence  to  make  a  decision),”  was  identified  in  interviews  after  the 
survey  had  been  distributed  to  a  portion  of  SMEs.  We  chose  to  keep  it  out  of  the  remaining  surveys  for 
consistency. 

4.2.3  Observed  Trends 

While  most  of  the  skills  identified  were  valued  and  described  consistently  by  all  types  of 
interviewees,  some  of  the  responses  strongly  correlated  to  the  role  that  the  professional  holds.  We 
observed  trends  between  rad/nuc  SMEs  and  emergency  managers  in  operational  decision  making  roles, 
trends  between  National  vs.  Regional  FEMA,  and  trends  between  federal  roles  overall  and  state/local 
roles  overall. 

Rad/Nuc  SMEs  vs.  Operational  Decision  Makers 

We  interviewed  a  range  of  rad/nuc  SMEs  with  extensive  knowledge  on  INDs  but  without  direct 
experience  in  operational  decision-making  roles,  including  IND  planners,  health  physicists,  and  public- 
health  professionals.  Their  assessment  of  decisions  and  skills  were  closely  coupled  to  those  in  operational 
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roles  in  most  regards,  suggesting  that  the  theoretical  and  practical  sides  of  the  community  are  in 
alignment.  In  one  aspect,  however,  the  two  groups  routinely  differed. 

Almost  every  decision  on  the  list  requires  some  understanding  of  INDs  or  radiation,  indicating  that 
all  SMEs  placed  importance  on  technical  knowledge.  However,  not  all  SMEs  called  it  out  as  an  area  of 
concern  in  its  own  right,  and  the  relative  emphasis  of  technical  knowledge  versus  other  skills  differed. 
The  rad/nuc  SMEs  tended  to  identify  a  lack  of  basic  technical  knowledge  about  radiation  among  their 
highest  concerns  with  current  preparedness.  Several  explicitly  said  that  they  trusted  that  the  people  filling 
emergency  management  roles  had  all  the  soft  skills  and  proficiencies  necessary,  but  they  believed  those 
emergency  managers  lacked  basic  familiarity  with  radiation  or  with  the  network  of  SMEs  available  to 
provide  such  information.  In  contrast,  emergency  managers  in  decision  making  roles  (of  all  levels)  were 
much  more  likely  to  emphasize  the  relative  importance  of  the  cognitive  and  interpersonal  skills  needed  to 
effectively  apply  the  technical  knowledge,  and  more  likely  to  downplay  the  presence  of  the  technical 
knowledge  as  a  critical  gap  (or,  at  least,  downplay  it  as  a  gap  that  is  hard  to  fill  with  traditional  training 
methods). 

Several  of  the  rad/nuc  SMEs  noted  that  the  necessary  technical  material  can  be  covered  in  a 
classroom,  but  they  worried  that  such  methods  did  not  result  in  the  knowledge  being  retained  since  the 
majority  of  responders  did  not  have  much  opportunity  to  practice  or  apply  IND-related  knowledge.  Their 
concerns  suggest  a  slightly  different  opportunity  that  augmented  training  techniques  might  support — 
improving  retention  of  technical  knowledge  by  providing  more  concrete  and  interactive  rehearsal  without 
a  full-scale  exercise. 

National  vs.  Regional  FEMA 

National  level  FEMA  professionals  consistently  emphasized  navigating  politics  carefully,  building 
cohesive  and  effective  teams,  and  being  aware  of  the  advisory  role  that  FEMA  plays  to  state  and  locals. 
Regional  FEMA  and  federal  domain  SMEs  consistently  emphasized  preparation  and  training,  trusting 
SMEs  over  political  advisors,  and  the  importance  of  making  good  decisions.  Table  7  provides  a 
comparison  between  typical  responses  from  the  regional  and  national  level  FEMA  responders. 
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Table  7 

Comparison  of  Regional  and  National  Level  FEMA  Responses 


Subject 

Typical  Regional  FEMA  Stance 

Typical  National  FEMA  Stance 

Decision-Making 

It  is  important  to  make  the  right 
decision  quickly,  as  the  early  stages 
of  the  incident  are  critical. 

It  is  important  to  maintain  an  advisory 
role  until  asked  to  do  otherwise. 

Basis  for  Strategic 
Decisions 

Strategic  decisions  should  be 
informed  by  factual  knowledge  and 
aimed  to  save  lives.  They  should  not 
be  swayed  by  political  factors,  which 
are  distractions. 

Strategic  decisions  need  to  be 
reflected  in  organizational  structures, 
and  take  into  account  the  social  and 
political  environment  for  them  to  be 
effective. 

Setting  Priorities 

Be  ready  to  make  hard  decisions  to 
maximize  the  ultimate  goal  of  saving 
lives. 

Priorities  differ  in  different  situations, 
and  it  is  important  to  understand  and 
communicate  your  goals. 

Key  Skills 

Decision-making  under  pressure, 
without  sufficient  information,  and  with 
no  perfect  options. 

Organizing  teams,  bringing  SMEs  to 
bear  on  the  appropriate  problem,  and 
creating  a  cohesive  response. 

Given  the  lengthy  experience  of  all  the  interviewees,  we  assume  that  the  discrepancies  shown  in 
Table  7  are  not  an  indication  of  one  group  being  right  or  more  knowledgeable,  and  that  the  two  stances 
are  not  contradictory  but  instead  differ  in  their  emphasis.  We  inteipret  the  differences  in  response  as 
reflecting  real  differences  in  the  skills  and  priorities  most  pertinent  to  different  types  of  emergency 
managers.  From  this  characterization,  we  can  start  to  form  a  picture  of  how  training  for  such  skills  might 
be  different  for  these  distinct  roles,  and  how  alternative  methods  may  be  required  to  meet  the  needs  of 
these  groups.  Recognizing  these  differences  could  be  especially  valuable  when  supporting  professionals 
moving  between  national  and  regional  roles. 

Federal  vs.  State/Local 

There  are  strong  similarities  in  how  the  federal  and  state/local  participants  prioritized  critical  skills. 
Many  of  the  skills  (e.g.,  timely  decisions,  resource  allocation,  and  selective  attention)  ranked  highly  by 
both  groups  were  described  in  nearly  identical  manners.  For  other  skills,  the  same  underlying  competency 
was  described  as  manifesting  in  different  ways,  reflective  of  the  varying  roles  local,  state,  and  federal 
emergency  managers  play. 

For  example,  for  the  skill  “ability  to  adapt  plans  and  improvise  solutions  based  on  changing 
conditions”  (ranked  3rd  among  cognitive  skills  in  the  interviews),  the  federal-level  emergency  managers 


41 


generally  emphasized  the  importance  of  adjusting  a  prepared  plan  as  the  incident  unfolds  and  improvising 
on  the  fly.  Conversely,  the  state  and  local  emergency  managers  generally  emphasized  adapting  a  plan  to 
the  particular  needs  of  an  agency  or  community  prior  to  an  incident.  We  tallied  both  responses  as  a  single 
skill,  defined  as  the  ability  to  interpret  a  plan’s  intent  and  decide  when  to  execute  it  as  written  and  when 
to  modify  it  based  on  incident  specifics.  We  believe  the  differences  are  worth  noting,  as  they  could  affect 
the  design  of  training  materials  or  exercises  targeting  the  same  skill  for  different  types  of  emergency 
managers. 

A  similar  pattern  was  observed  with  the  skills  “determine  reliable  information  and  sources”  (ranked 
2nd  among  cognitive  skills  in  the  interviews),  and  “experience  working  with  the  response  team  or 
network”  (ranked  1st  among  experiential  skills).  Both  skills  describe  the  importance  of  having  and 
knowing  how  to  request  the  network  of  SMEs  that  can  be  leveraged  for  technical  guidance  for  decision 
makers.  Federal  emergency  managers  generally  described  these  skills  in  terms  of  identifying  SMEs  and 
integrating  teams  under  the  pressure  of  an  incident.  State/local  emergency  managers  generally  described 
this  integration  in  terms  of  building  social  and  organizational  connections  prior  to  an  incident. 

Considering  the  roles  that  federal  vs.  state/local  emergency  managers  typically  play  in  an  incident 
and  the  types  of  incidents  they  typically  encounter,  these  differences  are  not  surprising.  Federal 
responders  work  in  support  of  state/local  teams  when  the  incident  is  abnormal  or  of  a  scale  that 
overwhelms  the  available  local  resources.  Federal  teams  may  be  called  on  to  support  a  wide  range  of 
incidents,  so  plans  and  networks  must  be  revised  and  learned  on  the  fly,  as  it  is  not  possible  to  rehearse 
with  every  possible  partner  and  scenario.  In  contrast,  the  state/local  teams  deal  with  more  routine 
incidents,  allowing  them  the  opportunity  to  collaborate  with  the  same  agencies.  In  such  cases,  it  is 
feasible  to  adapt  plans  and  build  networks  ahead  of  time  to  preempt  needs  that  arise  during  the  incident. 

If  anything,  the  surprising  result  is  not  that  the  two  groups  differ  in  their  expression  of  those  skills, 
but  that  their  different  needs  boil  down  to  such  similar  underlying  skills.  Hence,  similar  materials  can 
likely  be  used  to  augment  training  for  both  types  of  emergency  managers,  but  those  materials  will  need  to 
be  grounded  in  different  examples  and  exercise  scenarios. 
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5.  CONCLUSIONS  AND  NEXT  STEPS 


This  document  has  described  an  analysis  of  the  key  decision  points  and  corresponding  skills  where 
current  training  and  preparation  may  leave  emergency  managers  and  responders  unprepared  during  an 
IND  incident  or  other  large  scale  disaster.  Through  reviewing  documentation  and  conducting  interviews 
with  26  SMEs  and  survey  responses  from  1 1  SMEs,  we  can  better  understand  common  threads  in  the 
difficult  decisions  in  an  IND  incident  response  and  the  corresponding  skills  possessed  by  experienced 
responders  who  make  them.  Unique  challenges  and  decisions  exist  in  an  IND  incident  response,  but  the 
underlying  skills  identified  as  addressing  those  decisions  were  not  IND  specific,  and  would  likely  apply 
to  any  moderate  to  large-scale  incident.  We  observed  that  most  critical  decisions  are  made  especially 
difficult  by  time  pressure,  amplifying  the  importance  of  cognitive  and  interpersonal  skills.  If  time  were 
not  a  constraint,  then  all  of  the  requisite  information  could  be  gathered,  the  team  could  come  to  a 
consensus  on  each  issue  as  it  arose,  and  the  task  would  be  a  more  straight-forward  technical  question. 

When  we  look  at  the  decision-making  skills  emphasized  by  the  SMEs,  cognitive  and  interpersonal 
skills  were  highlighted  significantly  more  than  technical  knowledge  and  experience.  As  stated  in  the  skill 
analysis  section,  we  believe  that  this  is  because  all  responders  who  are  in  important  emergency  response 
positions  already  have  the  necessary  technical  knowledge  and  experience.  These  skills  are  effectively 
taught  and  selected  for  using  the  traditional  methods  of  classroom  and  on-the-job  training,  although  there 
are  concerns  that  retention  may  be  poor  for  skills  that  are  not  applied  in  practice  (such  as  IND-related 
knowledge).  The  cognitive  and  interpersonal  skills  emphasized  by  expert  emergency  responders  are  much 
more  difficult  to  quantify  and  teach.  Those  skills  are  currently  developed  only  over  years  of  experience, 
limiting  the  pool  of  qualified  emergency  managers  and  making  it  difficult  to  fill  all  the  decision-making 
positions.  We  see  this  gap  as  a  possibility  to  deliver  training  and  evaluation  of  cognitive  and  inteipersonal 
skills  through  the  use  of  serious  gaming,  and  to  improve  the  retention  of  relevant  technical  knowledge. 
Several  SMEs  expressed  the  sentiment  that  well-run  exercises  can  address  those  needs,  but  that  exercises 
are  a  time-consuming  way  of  accumulating  knowledge,  and  require  SMEs  to  be  taken  away  from  their 
posts  to  participate.  We  also  hope  to  explore  the  ability  for  techniques  and  technologies  from  gaming  to 
provide  a  less  burdensome  format  for  exercises,  or  as  augmentation  to  existing  exercise  regiments. 

A  central  concept  of  the  larger  effort  is  identifying  how  to  use  repeatable  and  interactive 
experiences  outside  the  field  of  radiological  and  nuclear  disaster  management  to  help  disaster 
management  professionals  develop  and  demonstrate  relevant  expertise  [3].  The  next  steps  in  the  project 
will  be  to  investigate  how  the  findings  from  this  initial  report  can  inform  the  development  of  training, 
evaluation,  and  research  platforms.  These  platforms  may  be  in  the  form  of  strategic  games  to  help 
individuals  and  teams  of  radiological  response  officials  and  emergency  managers  rapidly  improve 
pertinent  behavioral  competences,  or  rehearsal  exercises  augmented  with  gaming  techniques  to  impose 
reduced  burden  on  participants.  The  effort  will  explore  classifying  what  types  of  game  mechanics  are 
most  suited  to  the  different  types  of  skills  pertinent  to  emergency  management. 
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APPENDIX  A 

SUPPLEMENTAL  ISSUES  RAISED  BY  SMEs 


The  SME  interviews  were  focused  on  eliciting  information  about  key  decisions  and  skills.  A  topic 
was  only  included  in  the  analysis  if  at  least  two  SMEs  spontaneously  raised  the  issue.  However,  some 
information  of  note  that  came  out  of  the  interviews  does  not  meet  this  criterion,  either  because  it  does  not 
directly  pertain  to  a  specific  decision  or  skill,  or  because  it  was  only  raised  by  a  single  SME.  Below  are 
some  insights  provided  by  SMEs  that  do  not  directly  pertain  to  the  analysis,  but  helped  us  better 
understand  the  situation  facing  an  emergency  manager  and  the  types  of  training  techniques  that  might 
better  prepare  them.  We  believe  they  bear  repeating  despite  falling  outside  the  bounds  of  the  more  formal 
analysis,  as  these  ideas  may  help  design  new  training  materials  and  methods. 

Replicating  the  Political  Environment 

It  is  impossible  to  accurately  replicate  the  political  environment  of  a  real  disaster,  and  exercises 
may  exaggerate,  downplay,  or  distort  the  importance  and  nature  of  political  complications.  Political  issues 
may  be  exaggerated  during  exercises,  where  appearances  are  important  and  no  lives  are  at  stake,  but  fall 
to  the  background  during  an  actual  crisis.  However,  exercises  usually  involve  professionals  not 
politicians,  so  they  may  not  capture  some  political  issues  at  all.  For  example: 

•  How  willing  will  officials  be  to  conspicuously  make  no  attempt  to  rescue  some  portions  of  the 
effected  public? 

•  How  willing  will  they  be  to  conspicuously  prioritize  one  region’s  response  or  recovery  over 
another? 

•  Will  officials  present  a  unified  front  with  political  opponents  or  will  differences  produce  a 
mixed  message  to  the  public  and  undermine  confidence  in  the  response? 

The  organizational  structure  of  the  response  should  be  built  around  the  incident  not  brought  to  the 
incident,  e.g.,  with  a  well  understood  incident,  more  frequent  updates  may  be  more  appropriate  so  that 
communication  flows  freely.  However,  with  a  more  novel  incident,  fewer  updates  may  be  more 
appropriate  so  that  teams  have  uninterrupted  time  to  problem  solve. 

Moral  Dilemmas  Are  Not  Just  Distractions 

The  best  decision  can  be  a  matter  of  moral  and  ethical  philosophy  rather  than  a  technical  problem, 
or  a  matter  of  following  clear  priorities  with  discipline.  For  example,  bringing  in  medical  teams  instead  of 
search  and  rescue  teams  may  ultimately  save  more  lives — a  decision  that  appears  difficult  emotionally  but 
with  a  clear  right  answer  in  the  grand  scheme.  However,  prioritizing  search  and  rescue  teams  may 
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minimize  human  suffering  or  improve  public  morale  by  making  it  clear  that  the  responders  have  not  given 
up  on  any  part  of  the  affected  public.  Maintaining  confidence  in  the  government,  social  stability,  and  a 
sense  of  fair  justice  may  outweigh  total  lives  saved.  The  correct  moral  action  depends  on  the  underlying 
ethical  structure  and  larger  scale  priorities.  It  is  unclear  what  responsibility  responders  have  to  honor  the 
philosophy  implied  by  their  superiors’  stated  priorities  versus  acting  on  their  own  moral  judgment. 
Emphasizing  individual  responsibility  can  undermine  cohesion  of  the  response  effort,  but  emphasizing 
conformance  to  higher  priorities  can  undermine  flexibility  to  local  circumstances. 

While  it  was  not  a  direct  question  asked  of  the  interviewees,  many  voluntarily  discussed  their  views 
on  what  it  meant  to  have  a  successful  response  effort.  Saving  lives  was  always  listed  as  a  key  measure, 
but  it  was  not  always  listed  first.  Reducing  human  suffering,  maintaining  our  system  of  government, 
preparing  for  the  recovery  effort,  and  maintaining  good  relations  in  support  of  future  response  efforts 
were  also  mentioned  as  key  factors  rivaling  lifesaving  in  importance.  These  goals  often  align. 

Well  after  the  stabilization  period  of  a  disaster,  managing  economic  recovery  can  be  unintuitive  and 
present  further  moral  dilemmas.  Restoring  economic  integrity  can  be  central  to  restoring  normal  life,  but 
that  can  be  at  odds  with  addressing  humane  issues  first  and  can  violate  some  people’s  sense  of  fairness  or 
justice.  For  example,  should  rebuilding  efforts  initially  focus  on  hospitals  to  reduce  suffering  of  the 
injured;  middle-class  homes  representing  normality  to  the  bulk  of  the  population;  symbols  of  order  such 
as  city  hall;  or  casinos  that  provide  tax  revenue  to  revive  the  economy  and  fund  other  efforts? 
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GLOSSARY  OF  ACRONYMS 


CBRNE 

Chemical  Biological  Radiological  Nuclear  and  high-  yield  Explosives 

DoD 

Department  of  Defense 

DHS 

Department  of  Homeland  Security 

DHS S&T 

Department  of  Homeland  Security  Science  and  Technology  Directorate 

DZ 

Damage  Zone 

FEMA 

Federal  Emergency  Management  Agency 

FR 

First  Responder 

FZ 

Fallout  Zone 

ICS 

Incident  Command  System 

IMAT 

Incident  Management  Assistance  Team 

IND 

Improvised  Nuclear  Device 

NIMS 

National  Incident  Management  System 

NYC 

New  Y ork  City 

Rad/Nuc 

Radiological  or  Nuclear 

RDD 

Radiological  Dispersal  Device 

SAR 

Search  and  Rescue 

SME 

Subject  Matter  Expert 

STEP 

Shelter  and  Temporary  Emergency  Power 
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evaluation  of  emergency  managers  to  augment  and  complement  existing  techniques.  The  effort  seeks  to  answer  the  question  of  how 
government  agencies  can  ensure  that  key  emergency  response  personnel  have  the  required  skills  and  knowledge  to  make  critical  decisions 
during  an  incident  of  unprecedented  size,  scope,  and  complexity  such  as  an  IND  detonation. 

This  document  reports  on  findings  and  analysis  during  the  first  phase  of  the  project,  which  was  to  gain  an  understanding  of  the  specific 
response  decisions  and  decision  making  skills  required  during  an  IND  detonation  response.  The  project  team  interviewed  and  surveyed 
emergency  response  professionals,  analyzed  the  decisions  they  emphasized  as  being  both  important  and  difficult,  and  analyzed  the 
associated  skills  they  identified  as  being  critical  to  making  these  decisions.  The  remainder  of  the  report  provides  detail  on  the  method  and 
findings  of  these  interviews.  This  effort  does  not  intend  to  develop  an  exhaustive  list  of  all  relevant  decisions  and  skills.  Rather,  it  aims  to 
prioritize  key  skills  and  focus  attention  on  potential  gaps,  in  order  to  lay  the  groundwork  for  enhancements  to  training  and  evaluation 
techniques. 
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